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DEFENSE  TRANSPORTATION 

DOD  Needs  to  Improve  the  Evaluation  of  Safety  and 
Performance  Information  for  Carriers  Transporting 
Security-Sensitive  Materials 


Why  GAO  Did  This  Study 

In  fiscal  year  2014,  DOD  facilitated  the 
transport  by  commercial  motor  carriers 
(private-sector  trucking  companies)  of 
nearly  50,000  separate  shipments  of 
security-sensitive  material  (e.g., 
ammunition  and  explosives)  in  the 
continental  United  States.  DOD  uses 
DOT  safety  performance  information  to 
determine  whether  these  carriers  can 
transport  security-sensitive  material 
under  the  TPS  program. 

The  House  Report  113-446  included  a 
provision  for  GAO  to  assess  matters 
related  to  the  safety  performance, 
standards,  and  other  aspects  of  TPS 
carriers.  This  report  examines  the 
extent  to  which  (1)  DOD’s  use  of 
DOT'S  safety  performance  information 
results  in  sufficient  and  reliable 
information  for  DOD  to  evaluate  the 
safety  performance  of  individual  TPS 
carriers,  and  (2)  DOD  evaluates  TPS 
carrier  incident  data  and  investigates 
incidents  to  identify  safety-related 
risks.  GAO  analyzed  DOT  and  DOD 
safety  performance  data  on  the  55 
carriers  participating  in  the  TPS 
program  in  fiscal  years  2011-14. 

What  GAO  Recommends 

GAO  recommendations  include  that 
DOD  establish  an  approach  for 
reviewing  available  safety  violation 
data,  and  develop  guidance  on 
analyzing  incident  trends  and  fully 
investigating  incidents.  DOD  concurred 
with  the  recommendations  on 
analyzing  and  investigating  incidents 
but  did  not  concur  with  the  violations 
data  recommendation  because  DOD 
stated  that  it  does  not  own  the  data, 
the  data  do  not  distinguish  TPS  from 
non-TPS  shipments,  and  research  is 
needed  on  other  data.  GAO  believes 
the  recommendation  remains  valid. 

View  GAO-16-82.  For  more  information, 
contact  Cary  B.  Russell  at  (202)  512-5431  or 
russellc@gao.gov  or  Susan  A.  Fleming  at 
(202)  512-2834  orflemings@gao.gov. 


What  GAO  Found 

The  Department  of  Defense’s  (DOD)  use  of  the  Department  of  T ransportation’s 
(DOT)  safety  performance  information  does  not  fully  result  in  sufficient  and 
reliable  information  to  evaluate  the  safety  performance  of  individual  commercial 
motor  carriers  that  transport  security-sensitive  materials  under  the  Transportation 
Protective  Service  (TPS)  program.  DOT  uses  data  from  roadside  inspections, 
crash  investigations,  and  other  sources  to  assign  carriers  overall  Safety  Ratings 
and  relative  Safety  Measurement  System  (SMS)  scores.  The  SMS  scores  track 
safety  performance  in  several  areas  such  as  hazardous  materials  compliance 
and  vehicle  maintenance,  and  range  from  0  to  100,  where  100  indicates  the 
worst  relative  safety  performance.  From  November  2012  through  October  2014, 
DOD  maintained  TPS  carriers  with  absent  or  dated  DOT  Safety  Ratings. 
However,  DOD  conducts  its  own  inspections  of  TPS  carriers,  which  partially 
compensates  for  this  issue.  DOD  officials  told  GAO  that  all  inspected  carriers 
passed.  In  February  2014,  GAO  reported  that  SMS  scores  for  many  commercial 
carriers  may  not  be  reliable  indicators  of  safety  performance  because  they  were 
based  on  insufficient  information,  such  as  infrequent  inspections.  Consequently, 
GAO  recommended  that  DOT  revise  the  methodology  for  determining  these 
scores.  DOT  did  not  concur,  but  stated  that  it  would  continue  to  improve  the 
effectiveness  of  the  SMS  methodology.  In  this  review,  GAO  found  that  many  TPS 
carriers’  SMS  scores  were  subject  to  similar  limitations.  However,  DOD  is  not 
using  a  rich  source  of  publicly  available  violation  data  to  evaluate  TPS  carriers — 
data  that  include,  for  example,  violations  related  to  a  driver’s  use  of  controlled 
substances  while  transporting  hazardous  materials.  Absent  an  approach  to 
review  available  violation  data,  DOD  is  not  well-positioned  to  effectively  evaluate 
the  carriers  it  entrusts  to  transport  security-sensitive  material. 

DOD  collects  incident  data  involving  TPS  carriers  of  security-sensitive  materials 
but  does  not  evaluate  the  data  to  determine  whether  systemic  trends  and 
patterns  are  linked  to  safety  risks  and  does  not  fully  investigate  incidents  to 
determine  root  causes.  For  example,  mechanical  breakdown  incidents  have 
resulted  in  delays  and  increased  the  exposure  of  security-sensitive  material  to 
the  general  public.  DOD  has  not  analyzed  TPS  carrier  incident  information  to 
identify  any  trends  because  there  is  no  guidance  requiring  it  to  do  so.  GAO’s 
analysis  of  749  mechanical  breakdown  incidents  from  fiscal  years  2011  to  2014 
found  that  98  percent  of  the  TPS  carriers  that  had  mechanical  breakdowns  while 
transporting  the  highest  risk  materials  such  as  stinger  missiles  and  grenades 
stopped  for  more  than  2  hours.  Further,  in  some  cases  such  as  when  a  fire 
occurs,  DOD  may  conduct  an  investigation,  but  DOD  does  not  generally  conduct 
full  investigations  of  security-sensitive  shipments  because  there  is  no 
department-wide  guidance  that  requires  comprehensive  investigations  of 
incidents  involving  commercial  carriers  with  security-sensitive  materials.  Also,  for 
3  of  the  4  incidents  DOD  investigated  since  fiscal  year  2012,  DOD  did  not  identify 
the  root  causes  to  prevent  recurrences  due  to  the  lack  of  department-wide 
guidance.  Without  department-wide  guidance  requiring  the  evaluation  of  TPS 
incident  data  to  identify  trends  and  the  investigation  of  incidents  including 
determination  of  the  root  causes,  it  will  be  difficult  for  DOD  to  consistently  and 
effectively  identify  safety  risks  that  could  help  reduce  future  incidents. 
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Abbreviations 


BASIC 

Behavior  Analysis  and  Safety  Improvement 
Categories 

CSA 

Compliance,  Safety,  Accountability 

DOD 

Department  of  Defense 

DOT 

Department  of  Transportation 

DTTS 

Defense  Transportation  Tracking  System 

FCRP 

Freight  Carrier  Registration  Program 

FMCSA 

Federal  Motor  Carrier  Safety  Administration 

HAZMAT 

Hazardous  Materials 

MFTURP-1 

Military  Freight  Traffic  Unified  Rules  Publication-1 

TRANSCOM 

U.S.  Transportation  Command 

SRC 

Security  Risk  Category 

SDDC 

Surface  Deployment  and  Distribution  Command 

TPS 

Transportation  Protective  Service 

VI N 

Vehicle  Identification  Numbers 
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The  Honorable  John  McCain 
Chairman 

The  Honorable  Jack  Reed 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Mac  Thornberry 
Chairman 

The  Honorable  Adam  Smith 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

In  fiscal  year  2014,  the  Department  of  Defense  (DOD)  facilitated  the 
transportation  of  nearly  50,000  separate  shipments  of  security-sensitive 
materials — missiles,  ammunition,  explosives,  weapons,  and  other 
sensitive  materials — throughout  the  continental  United  States  by 
commercial  motor  carriers  (“carriers”).  These  carriers  are  private-sector 
trucking  companies  that  are  hired  to  ship  materials  for  DOD.  Trucks 
carrying  these  shipments  travel  millions  of  miles  on  U.S.  interstates, 
highways,  and  local  thoroughfares  across  the  continental  United  States 
and  Alaska.1  Some  of  these  shipments  (e.g.,  explosives  and  weapons) 
can  be  high  risk  and,  if  unsafely  transported,  could  pose  a  significant 
threat  to  transportation  workers,  emergency  responders,  and  the  general 
public.  The  Department  of  Transportation’s  (DOT)  Federal  Motor  Carrier 
Safety  Administration  (FMCSA)  is  to  ensure  safe  motor  carrier  operations 
by  enforcing  safety  regulations  and  targeting  high-risk  carriers,  in  part  by 
using  data  obtained  from  federal  and  state  roadside  inspections  and 
crash  investigations.  FMCSA’s  Compliance,  Safety,  Accountability  (CSA) 
program  uses  these  data  to  assign  safety  performance  scores  to  identify 
high-risk  carriers  for  subsequent  enforcement  activities,  such  as  warning 


1  In  Hawaii  security-sensitive  material  arrive  on  military  planes  or  on  ships  and  are 
transported  by  the  Department  of  Defense’s  military  vehicles. 
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letters  or  investigations.  FMCSA  also  uses  these  data,  along  with  other 
information,  to  assign  carriers  overall  Safety  Ratings. 

Nearly  90  percent  of  DOD’s  security-sensitive  shipments  are  transported 
using  commercial  motor  carriers.  Within  DOD,  the  U.S.  Transportation 
Command  is  a  unified  and  functional  combatant  command  charged  with 
providing  transportation  support  to  the  other  combatant  commands, 
military  services,  and  defense  organizations.  The  U.S.  Transportation 
Command  manages  the  use  of  commercial  motor  carriers  through  the 
Army’s  Surface  Deployment  and  Distribution  Command  (SDDC)  and  that 
command  oversees  the  Transportation  Protective  Service  (TPS)  program, 
which  provides  the  ground  transport  of  security-sensitive  materials.  SDDC 
uses  safety  performance  information  from  DOT  as  part  of  the  command’s 
determination  of  whether  a  carrier  is  qualified  to  join  or  remain  in  the  TPS 
program.  According  to  SDDC  officials,  from  fiscal  year  201 1  through  fiscal 
year  2014,  SDDC  contracted  with  55  commercial  motor  carriers  to 
participate  in  the  TPS  program.  TPS  carriers  follow  additional  procedures 
and  meet  more  stringent  safety  and  security  standards  than  other 
commercial  motor  carriers.  For  example,  Chapter  205  of  the  Defense 
Transportation  Regulation  requires  that  TPS  carriers  handling  certain 
security-sensitive  shipments  are  to  have  two  drivers  with  security 
clearances,  and  one  of  the  drivers  must  stay  with  the  shipment  at  all 
times. 2  DOD  is  able  to  track  the  movement  of  TPS  carriers  transporting 
security-sensitive  materials  in  the  U.S.  Transportation  Command’s 
Defense  Transportation  Tracking  System.  Drivers  involved  in  incidents 
(mechanical  breakdowns,  accidents,  or  suspicious  activities)  are  to 
contact  the  Defense  Transportation  Tracking  System  staff  to  trigger  an 
emergency  response.  The  Army’s  SDDC  leads  DOD  efforts  in  responding 
to  incidents  involving  DOD’s  security-sensitive  materials. 

In  February  2014,  we  examined  the  effectiveness  of  DOT’S  CSA  program 
in  assessing  the  safety  risk  of  commercial  motor  carriers,  and 
recommended  that  DOT  revise  the  methodology  for  determining  the 
CSA’s  Safety  Measurement  System  scores  (calculated  using  CSA’s 
safety  performance  data)  to  account  for  limitations  in  comparing  safety 


2  See  Department  of  Defense,  Defense  Transportation  Regulation  4500. 9-R,  pt.  II,  ch. 
205,  Transportation  Protective  Service  (TPS),  para.  0.2  (July  27,  2015). 
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performance  data  across  commercial  motor  carriers.3  DOT  did  not  concur 
with  our  recommendation,  but  stated  that  it  would  analyze  our 
recommendations  as  DOT  continued  to  improve  the  effectiveness  of  the 
Safety  Measurement  System.  In  addition,  in  May  2014,  we  recommended 
that  DOD  take  actions  to  improve  the  documentation  of  hazardous 
materials,  eliminate  the  secure-hold  denial4  of  hazardous  materials 
shipments,  and  examine  the  limitations  on  CSA  data  used  to  select  TPS 
carriers  of  hazardous  materials.  DOD  concurred  with  these 
recommendations  and,  in  November  2014,  established  a  secure-hold 
denial  category  in  the  Defense  Transportation  Tracking  System  for 
tracking  unauthorized  parking  of  security-sensitive  shipments  in 
response. 

The  House  Armed  Services  Committee  Report  113-446  included  a 
provision  for  GAO  to  assess  matters  related  to  the  safety  performance, 
standards,  and  other  aspects  of  commercial  motor  carriers  under  the  TPS 
program.5  Thus,  we  examined  the  extent  to  which  (1)  DOD’s  use  of 
DOT’S  safety  performance  information  results  in  sufficient  and  reliable 
information  for  DOD  to  evaluate  the  safety  performance  of  individual  TPS 
carriers,  and  (2)  DOD  has  evaluated  TPS  carrier  incident  data  and 
investigated  TPS  carrier  incidents  to  identify  safety-related  trends.  For  the 
purposes  of  our  review,  we  use  the  term  “incident”  to  refer  to  a  crash, 
mechanical  breakdown,  or  suspicious  activity. 

To  address  our  first  objective,  we  reviewed  DOD  guidance  regarding 
DOD’s  procedures  for  evaluating  the  safety  of  TPS  carriers  (i.e.,  for  them 


3  FMCSA  refers  to  Safety  Measurement  System  scores  as  SMS  “percentiles,”  but  for  the 
purposes  of  this  report  we  refer  to  them  as  SMS  “scores.”  GAO,  Federal  Motor  Carrier 
Safety:  Modifying  the  Compliance,  Safety,  Accountability  Program  Would  Improve  the 
Ability  to  Identify  High  Risk  Carriers,  GAO-14-1 14  (Washington,  D.C.:  Feb.  3,  2014). 

4  GAO,  DOD  Needs  to  Take  Actions  to  Improve  the  Transportation  of  Hazardous  Material 
Shipments,  GAO-14-375  (Washington,  D.C.:  May  1, 2014).  A  “secure-hold”  area  is  a 
location  designated  for  the  temporary  parking  of  commercial  motor  carrier  vehicles 
transporting  Department  of  Defense  (DOD)-owned  ammunition  and  explosives,  and  other 
security-sensitive  material.  According  to  the  Defense  Transportation  Regulation,  DOD 
installations  are  to  assist  commercial  motor  carriers  transporting  DOD  shipments  of  arms, 
ammunition,  and  explosives  by  providing  secure-hold  areas  in  the  interest  of  public  safety 
and  national  security  or  by  routing  the  shipments  to  the  nearest  location  that  has  a  secure- 
hold  area.  A  denial  is  when  carriers  transporting  DOD  security-sensitive  materials  are  not 
provided  access  to  secure  hold  areas  within  DOD  installations. 

5  See  H.R.  Rep.  No.  113-446,  at  116-17  (2014). 
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to  enter  and  remain  in  the  TPS  program)  and  findings  from  our  February 
2014  report.6  We  assessed  DOD  guidance  against  Standards  for  Internal 
Control  in  the  Federal  Government.7  We  also  reviewed  DOT  safety 
performance  information,  including  Safety  Measurement  System  scores 
and  Safety  Ratings.  We  reviewed  DOT  safety  performance  information 
available  for  the  55  commercial  motor  carriers  who  participated  in  the 
TPS  program  from  fiscal  year  201 1  through  fiscal  year  2014,  which  were 
the  most  recent  data  available.  To  determine  the  extent  to  which  the 
limitations  we  had  reported  on  in  February  2014  regarding  Safety 
Measurement  System  scores  applied  specifically  to  individual  TPS 
carriers,  we  obtained  and  analyzed  the  most  recently  available  2-year 
“snapshot”  of  CSA  safety  performance  data  (November  2012  through 
October  2014).  The  2-year  snapshots  are  the  basis  for  DOT’S  score 
calculations.  During  this  period,  2  of  the  55  TPS  carriers  ceased  activity; 
therefore,  we  excluded  those  carriers  from  any  analyses  that  required 
observations  over  the  entire  2-year  period.  In  addition,  we  compared  TPS 
carriers’  Safety  Measurement  System  scores  in  this  2-year  snapshot  with 
those  DOD  requires  carriers  to  maintain  to  remain  in  the  TPS  program. 
We  were  unable  to  compare  TPS  carriers’  Safety  Measurement  System 
scores  at  entry  into  the  TPS  program  with  those  DOD  requires  of  carriers 
to  enter  the  TPS  program  because  the  CSA  program — including  Safety 
Measurement  System  scores — was  not  implemented  until  2010,  and  most 
TPS  carriers  had  entered  the  program  before  then.  To  evaluate  DOD’s 
use  of  Safety  Ratings,  which  are  comprehensive  assessments  of  a 
carrier’s  compliance  with  safety  fitness  standards  established  in  DOT 
regulations  that  FMCSA  conducts  periodically,  we  first  examined  the  most 
recent  Safety  Rating  data  available  for  all  55  TPS  carriers.  We  compared 
DOD’s  use  of  DOT  safety  performance  information  with  GAO’s  standards 
for  data  reliability8  and  with  Standards  for  Internal  Controls  in  the  Federal 
Government.  In  addition,  we  met  with  officials  from  DOD’s  U.S. 
Transportation  Command  to  identify  key  components  of  the  TPS  program 
and  with  officials  from  the  Department  of  Energy’s  Environmental 


6  GAO-14- 1 14. 

7GAO,  Standards  for  Internal  Control  in  the  Federal  Government,  GAO/AIMD-OO-21.3.1 
(November  1999).  These  state  that  management  should  use  relevant,  reliable,  and  timely 
information  and  that  policies  and  procedures  help  ensure  that  management’s  directives 
are  carried  out. 

8GAO,  Assessing  the  Reliability  of  Computer-Processed  Data,  GAO-09-365G  (February 
2009). 
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Management  Program  to  compare  their  program  that  uses  commercial 
motor  carriers  to  transport  some  hazardous  materials  with  the  TPS 
program.  We  met  with  officials  from  the  American  Trucking  Associations, 
to  identify  information  on  safety  data  or  standards  recommended  or  used 
by  other  agencies  and  organizations  knowledgeable  about  the  transport 
of  security-sensitive  materials.  We  determined  that  the  DOT  safety 
performance  information  was  sufficiently  reliable  for  the  purpose  of  our 
data  analysis  by  reviewing  relevant  DOT  documentation  on  Safety 
Measurement  System  methodology;  performing  standard  electronic  data 
reliability  testing,  such  as  looking  for  outliers  or  missing  data  in  the  2012- 
14  data  for  TPS  carriers;  and  interviewing  FMCSA  officials  about  any 
differences  in  Safety  Measurement  System  methodology  or  additional 
data  for  hazardous  materials  carriers.  Specifically,  the  purpose  of  our 
analysis  was  to  determine  the  extent  to  which  limitations  in  Safety 
Measurement  System  methodology  we  identified  in  February  2014 
applied  to  all  55  TPS  carriers.  Determining  that  the  data  are  reliable  for 
the  purposes  of  this  report  does  not  mean  that  the  data  are  reliable  in 
general,  or  for  other  uses,  such  as  comparing  safety  performance  across 
carriers. 

To  address  our  second  objective,  we  compared  safety  procedures  from 
DOD  Instruction  6055.16,  Explosives  Safety  Management  Program,  with 
our  analysis  of  1,039  incident  reports  from  the  Defense  Transportation 
Tracking  System  from  fiscal  year  201 1  through  fiscal  year  2014,  which 
were  the  most  recent  available  data,  to  determine  what  information  DOD 
collects,  reports,  and  tracks  when  there  is  an  incident  (i.e.,  an  accident, 
mechanical  breakdown,  or  suspicious  activity)  reported  through  the 
Defense  Transportation  Tracking  System  involving  TPS  carriers.  In 
addition,  we  compared  DOD’s  procedures  for  analyzing  TPS  carrier 
incident  data  and  investigating  TPS  carrier  incidents  with  DOD  Instruction 
6055.16,  which  emphasizes  the  importance  of  conducting  appropriate 
analyses  of  incidents  involving  DOD  ammunitions  and  explosives  in 
preventing  future  incidents.  We  interviewed  officials  from  the  Office  of  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics; 
the  DOD  Explosives  Safety  Board;  the  U.S.  Transportation  Command; 
the  Army  Headquarters  Safety  Office;  and  SDDC,  which  responds  to 
incidents  involving  DOD’s  security-sensitive  materials.  We  assessed  the 
reliability  of  the  Defense  Transportation  Tracking  System  data  and 
determined  the  data  we  examined  were  sufficiently  reliable  for  identifying 
challenges  and  the  extent  to  which  they  impact  the  transport  of  security- 
sensitive  materials.  We  sent  a  questionnaire  to  the  Defense 
Transportation  Tracking  System  officials  who  maintain  and  provide 
oversight  over  the  system  about  the  reliability  of  the  data  as  well  as  other 
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internal  and  quality  controls  in  place.  We  followed  up  with  interviews  as 
needed  to  clarify  any  responses.  We  also  requested  and  reviewed 
documentation  about  how  the  system  is  structured,  the  data  dictionary, 
and  written  procedures  in  place  to  ensure  that  the  appropriate  information 
in  the  Defense  Transportation  Tracking  System  is  collected.  Based  on  our 
assessment  of  the  reliability  of  the  Defense  Transportation  Tracking 
System  data,  we  determined  that  the  data  we  examined  were  sufficiently 
reliable  for  identifying  challenges  and  the  extent  to  which  they  impact  the 
transport  of  security-sensitive  materials. 

We  conducted  this  performance  audit  from  July  2014  to  December  2015 
in  accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives.  See  appendix  I  for  a  more 
detailed  explanation  of  our  scope  and  methodology. 


Background 


DOT  Safety  Performance  The  primary  mission  of  DOT’S  FMCSA  is  to  reduce  crashes,  injuries,  and 

Information  fatalities  involving  large  trucks  and  buses.  FMCSA  is  responsible  for 

overseeing  matters  related  to  the  safety  performance  of  the  commercial 
motor  carrier  industry.  As  part  of  this  role,  FMCSA  partners  with  state 
agencies  to  perform  roadside  inspections  of  commercial  motor  carriers’ 
vehicles  and  provide  information  from  these  inspections  and  crashes  that 
involve  commercial  motor  carriers  to  FMCSA. 

FMCSA  uses  information  from  on-site  reviews  known  as  “compliance 
reviews”  (i.e.,  assessments  of  carrier  compliance  with  DOT  safety  fitness 
standards)  to  assign  carriers  Safety  Ratings,  which  are  assessments  of 
the  overall  safety  of  a  carrier.  FMCSA  also  uses  information  from 
roadside  inspections  and  crashes  to  assign  carriers  “Safety  Measurement 
System”  scores,  assessments  of  carriers’  relative  safety  performance. 
Both  measures  capture  information  across  a  carrier’s  entire  operations. 
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During  compliance  reviews  FMCSA  determines  the  extent  to  which 
carriers  comply  with  safety  fitness  standards  established  in  DOT 
regulations.9  DOT  officials  noted  that  Safety  Ratings  do  not  expire. 
Compliance  reviews  include  interviews  with  carrier  officials  and  reviews  of 
carrier  records  that  pertain,  as  applicable,  to  the  alcohol  and  drug  testing 
of  drivers;  insurance  coverage;  crashes;  driver  qualifications;  the  number 
of  hours  a  driver  may  work;  vehicle  maintenance  and  inspections;  and  the 
transportation  of  hazardous  materials,  among  other  things.  Carriers  are 
assigned  a  Safety  Rating  of  “Satisfactory”  (i.e.,  adequate  safety 
management  controls  to  meet  safety  fitness  standard),  “Conditional”  (i.e., 
inadequate  safety  management  controls,  which  could  result  in  violations, 
crashes,  or  other  occurrences,  such  as  the  use  of  unqualified  drivers  or 
the  inadequate  maintenance  of  vehicles),  or  “Unsatisfactory”  (i.e., 
inadequate  safety  management  controls,  which  has  resulted  in  violations, 
crashes,  or  other  occurrences,  such  as  the  use  of  unqualified  drivers). 

Additionally,  using  Safety  Measurement  System  scores  as  part  of  the 
CSA  program,  FMCSA  ranks  carriers’  relative  safety  performance  in 
seven  safety-related  categories,  known  as  Behavior  Analysis  and  Safety 
Improvement  Categories  or  BASICS,  which  are  generally  assessments  of 
a  carrier’s  violations  in  that  category.  According  to  FMCSA,  the  BASICS 
have  been  developed  under  the  premise  that  commercial  motor  vehicle 
crashes  can  be  traced  to  the  behavior  of  motor  carriers  and  vehicle 
drivers.  The  seven  BASICS  are: 

•  Unsafe  Driving- related  to  the  operation  of  commercial  motor  vehicles 
in  a  dangerous  or  careless  manner; 


9  To  meet  the  safety  fitness  standard,  a  carrier  must  demonstrate  that  it  has  adequate 
safety  management  controls  in  place,  which  function  effectively  to  ensure  acceptable 
compliance  with  applicable  safety  requirements.  See  49  C.F.R.  §  385.5.  The  Federal 
Motor  Carrier  Safety  Administration  (FMCSA)  presently  conducts  compliance  reviews 
primarily  on  carriers  it  deems  high-risk,  meaning  that  they  have  exceeded  certain  Safety 
Measurement  System  thresholds  in  a  specific  combination  of  at  least  two  or  more  safety- 
related  categories  described  below.  More  specifically,  FMCSA  categorizes  a  cameras 
high-risk  if  either  its  Safety  Measurement  System  score  is  equal  to  or  greater  than  85  in 
the  Crash  Indicator,  Unsafe  Driving,  or  Hours-of-Service  Compliance  Behavior  Analysis 
and  Safety  Improvement  Category  and  the  carrier  is  at  or  above  certain  Safety 
Measurement  System  thresholds  in  one  other  category,  or  if  its  Safety  Measurement 
System  score  is  equal  to  or  greater  than  the  thresholds  of  four  or  more  of  any  of  the 
categories.  DOT,  Compliance ,  Safety ,  Accountability  (CSA)  Phase  I  and  2  Implementation 
Package  (Feb.  29,  2012).  DOT  has  been  planning  a  rulemaking  to  change  the  way  it 
assigns  Safety  Ratings  to  carriers  for  several  years. 
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•  Hours-of-Service  Comp//'ance-related  to  the  operation  of  commercial 
motor  vehicles  by  drivers  who  are  ill,  fatigued,  or  in  violation  of  the 
hours-of-service  regulations; 

•  Crash  Indicator — related  to  histories  or  patterns  of  high  crash 
involvement,  including  frequency  and  severity; 

•  Driver  Fitness — related  to  the  operation  of  commercial  motor  vehicles 
by  drivers  who  are  unfit  to  operate  a  commercial  motor  vehicle  due  to 
lack  of  training,  experience,  or  medical  qualifications; 

•  Controlled  Substances/Alcohol — related  to  the  operation  of  a 
commercial  motor  vehicle  by  drivers  who  are  impaired  due  to  alcohol, 
illegal  drugs,  and  misuse  of  prescription  or  over-the-counter 
medications; 

•  Vehicle  Maintenance — related  to  a  failure  to  properly  maintain  a 
commercial  motor  vehicle  and  prevent  shifting  loads;  and 

•  Hazardous  Materials  Compliance — related  to  unsafe  handling  of 
hazardous  materials  on  a  commercial  motor  vehicle. 

To  determine  the  Safety  Measurement  System  score  for  a  carrier, 

FMCSA  (1)  establishes  the  rate  at  which  carriers  violate  regulations 
related  to  the  BASICS,  which  involves  weighting  each  violation  and  crash 
on  the  basis  of  their  severity  and  on  when  these  events  occurred;  (2) 
identifies  whether  carriers  meet  minimum  data  sufficiency  standards  (e.g., 
minimum  number  of  violations  or  inspections  required  to  calculate  a 
Safety  Measurement  System  score);  and  (3)  compares  a  carrier  with 
others  with  a  similar  number  of  safety-related  “events”  (e.g.,  inspections, 
inspections  with  violations,  or  crashes  in  the  past  2  years).  FMCSA 
calculates  a  Safety  Measurement  System  score  by  ranking  carriers’ 
respective  violation  rates  (obtained  in  step  1  above)  and  assigning  each 
carrier  a  percentile  score,  ranging  from  0  to  100,  where  100  indicates  the 
highest  measure  and  worst  safety  performance  relative  to  other  carriers 
with  a  similar  number  of  safety-related  events  (calculated  in  step  3 
above).  If  a  carrier  does  not  meet  minimum  data  sufficiency  standards  for 
a  particular  BASIC  (step  2  above),  FMCSA  does  not  publish  a  score  for 
that  carrier  in  that  BASIC.  If  a  carrier  exceeds  FMCSA’s  defined 
thresholds  for  each  BASIC,  ranging  from  60  to  80  percent  for  carriers 
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transporting  hazardous  materials,  FMCSA  may  target  that  carrier  for  an 
intervention  (e.g.,  a  warning  letter  or  additional  investigations).10 

In  February  2014,  we  reported  that  the  CSA  program  faces  challenges  in 
reliably  assessing  the  relative  safety  risk  for  the  majority  of  commercial 
motor  carriers.11  Specifically,  we  found  the  following: 

•  Safety  Measurement  System  scores  are  poor  indicators  of  the 
propensity  of  individual  carriers  to  be  involved  in  a  future  crash.  For 
Safety  Measurement  System  scores  to  be  effective  indicators  for 
identifying  carriers  more  likely  to  crash,  the  violations  that  FMCSA 
uses  to  calculate  the  scores  should  have  a  strong  predictive 
relationship  with  crashes;  however,  we  found  that  most  regulations 
are  not  violated  frequently,  and  those  that  were  are  not  strongly 
associated  with  crash  risk. 

•  Most  carriers  lack  sufficient  safety  information  to  ensure  that  FMCSA 
can  reliably  compare  them  with  other  carriers,  a  key  component  of 
calculating  the  relative  Safety  Measurement  System  scores.  This  is 
especially  true  for  carriers  that  operate  fewer  vehicles  and  are 
inspected  infrequently.12 

Thus,  as  discussed  earlier  in  this  report,  we  recommended  that  DOT 
revise  its  methodology  for  determining  the  CSA’s  Safety  Measurement 
System  scores  to  account  for  limitations  in  comparing  safety  performance 
data  across  commercial  motor  carriers.  DOT  did  not  concur  with  our 
recommendation  but  stated  that  it  would  analyze  our  recommendations 
as  DOT  continued  to  improve  the  effectiveness  of  the  Safety 
Measurement  System. 


i  n 

The  CSA  program  has  three  main  components:  SMS,  interventions,  and  a  future  safety 
fitness  determination  rule.  As  of  October  2015,  DOT  officials  estimated  DOT  would  issue  a 
Notice  of  Proposed  Rulemaking  on  the  safety  fitness  determination  component  in  October 
2015. 

11  GAO-14-114. 

12For  example,  in  our  February  2014  report,  we  found  that  about  two-thirds  of  active 
carriers  operated  fewer  than  four  vehicles  and  that  these  carriers’  vehicles  received  few 
inspections.  For  instance,  carriers  operating  three  vehicles  received,  on  average,  about 
six  inspections  over  a  2-year  period.  (See  GAO-14-114). 
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DOD’s  Use  of  DOT  Safety  DOD  uses  two  pieces  of  DOT’S  safety  performance  information — Safety 
Performance  Information  Ratings  and  Safety  Measurement  System  scores — to  determine  which 

carriers  can  enter  and  remain  in  the  TPS  program.  DOD  requires  that 
TPS  carriers  have  a  Satisfactory  Safety  Rating  to  enter  the  TPS  program 
and  that  they  maintain  this  rating  to  remain  in  the  TPS  program.13  DOD 
also  requires  that  carriers  meet  certain  Safety  Measurement  System 
score  thresholds.14  To  remain  in  the  TPS  program,  TPS  carriers  are 
required  to  maintain  scores  at  or  below  75  for  the  Vehicle  Maintenance, 
Controlled  Substances/Alcohol,  Driver  Fitness,  and  Hazardous  Materials 
Compliance  categories;  and  at  or  below  60  for  the  Unsafe  Driving,  Hours- 
of-Service  Compliance,  and  Crash  Indicator  categories. 


DOD’s  Defense 
Transportation  Tracking 
System 


DOD  facilitates  the  transportation  of  security-sensitive  materials  including 
arms,  ammunition  and  explosives,  as  well  as  missiles,  weapons, 
classified  and  other  security-sensitive  materials.  DOD’s  security-sensitive 
materials  are  categorized  into  four  security  risk  categories  (SRC):  SRC  I, 
II,  III,  and  IV.  SRC  I  and  II  materials  are  the  most  sensitive  materials, 
while  SRC  III  and  IV  materials  are  the  least  sensitive.  Table  1  contains 
examples  of  materials  in  the  four  security  risk  categories  based  on 
information  from  Chapter  205  of  the  Defense  Transportation  Regulation15 
and  DOD  Manual  5100.76,  Physical  Security  of  Sensitive  Conventional 
Arms,  Ammunition,  and  Explosives.'16 


13Military  Freight  Traffic  Unified  Rules  Publication-1  (MFTURP-1)  (June  3,  2013);  SDDC, 
Freight  Carrier  Registration  Program  Welcome  Package  (Aug.  2014  and  Sept.  2013 
versions).  The  MFTURP-1  indicates  that  interstate  providers  must  maintain  a  Satisfactory 
Safety  Rating  with  FMCSA,  while  intrastate  providers  must  maintain  a  Satisfactory  Safety 
Rating  with  the  applicable  state  agency. 

14See  id. 

15  Department  of  Defense,  Defense  Transportation  Regulation  4500. 9-R,  pt.  II,  ch.  205, 
Transportation  Protective  Service  (TPS)  (July  27,  201 5). 

16  Department  of  Defense  Manual  5100.76,  Physical  Security  of  Sensitive  Conventional 
Arms,  Ammunition,  and  Explosives  (AA&E)  (Apr.  17,  2012). 
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Table  1 :  Examples  of  Security-Sensitive  Materials  by  Security  Risk  Category  (SRC) 

Security 

Risk  Category 

Arms,  missiles  and  rockets3 

Ammunition  and  explosives 

1 

Stinger  missiles,  javelin,  light  antitank  weapons,  and 
shoulder-launched  multi-purpose  assault  weapon 
rocket. 

Explosive  complete  rounds  for  SRC  1  missiles  and  rockets. 

II 

Ml 6,  M4  rifles,  light  automatic  weapons  such  as 

40mm  MK19  machine  guns,  and  weapon 
components  such  as  silencers,  mufflers,  and  noise 
suppression  devices. 

Hand  or  rifle  grenades,  antitank  or  antipersonnel  mines, 
explosives  used  in  demolition  operations,  warheads  used 
for  sensitive  missiles  and  rockets  weighing  fewer  than  50 
pounds. 

III 

Launch  tube  for  Stinger  missile,  tracker  for  dragon 
missiles,  mortar  tubes,  grenade  launchers,  rocket  and 
missile  launchers  below  a  certain  weight,  and  flame 
throwers. 

Ammunition  with  explosive-filled  projectile,  incendiary 
grenades,  blasting  caps,  supplementary  charges,  bulk 
explosives,  detonating  cord,  and  warheads  for  sensitive 
missiles  and  rockets  weighing  more  than  50,  but  less  than 
100  pounds. 

IV 

Shoulder-fired  weapons  such  as  shotguns  and  bolt 
action  rifles,  handguns,  and  recoilless  rifles. 

Ammunition  with  non-explosive  projectiles,  fuses  (other 
than  for  high  explosives  grenades),  incendiary  destroyers, 
and  riot  control  agents. 

Source:  The  Defense  Transportation  Regulation  and  Department  of  Defense  (DOD)  Manual  5100.76  |  GAO-16-82 


Note:  DOD  also  transports  other  security-sensitive  materials  that  are  not  categorized  in  the  four 
Security  Risk  Categories  (SRC  I  -  IV).  Examples  include  cryptographic  items  and  Navy  submarine 
class  688  propellers. 

aThere  are  no  arms  under  SRC  I. 


Each  year,  the  Defense  Transportation  Tracking  System  generates 
approximately  400  incident  reports,  tracks  between  150  and  275 
shipments  per  day,  and  processes  over  50,000  messages  per  month 
regarding  the  status  of  these  transported  shipments.  The  primary  function 
of  the  Defense  Transportation  Tracking  System  is  to  ensure  the  safe  and 
secure  transport  of  the  ammunition,  explosives,  and  other  security- 
sensitive  material  that  TPS  carriers  transport  in  the  continental  United 
States  and  Alaska.  DOD  uses  the  system’s  satellite-monitoring 
technology  and  a  24-hour  operation  center  to  track  the  transport  of 
security-sensitive  materials  and  also  to  report  and  facilitate  the  immediate 
emergency  response  to  TPS  carriers  involved  in  incidents  while 
transporting  security-sensitive  materials.  Using  satellite-based  monitoring 
technology,  DOD  can  track  TPS  carriers  from  the  time  they  pick  up  a 
shipment  through  the  point  at  which  carriers  deliver  the  shipment  to  the 
final  destination. 

The  Defense  Transportation  Tracking  System  collects  information  on 
incidents  involving  TPS  carriers  transporting  security-sensitive  materials. 
In  the  event  of  an  emergency  such  as  an  accident  or  a  suspicious  activity, 
a  TPS  driver  can  activate  an  immediate  emergency  response  by  hitting  a 


Page  11 


GAO-16-82  Defense  Transportation 


panic  button  to  alert  the  Defense  Transportation  Tracking  System  office. 
Depending  on  the  severity  of  the  situation,  the  Defense  Transportation 
Tracking  System  office  may  notify  the  local  police  or  DOD  organizations 
such  as  U.S.  Transportation  Command  orSDDC.  Non-emergency 
incidents  such  as  mechanical  breakdowns,  secure  hold  denials,  or 
unusual  delays  such  as  unauthorized  stops  are  also  reported  to  the 
Defense  Transportation  Tracking  System  by  the  driver  or  the  commercial 
motor  carrier.  For  both  emergency  and  non-emergency  situations,  the 
Defense  Transportation  Tracking  System  office  generates  an  incident 
report.  The  incident  report  contains  information  such  as  the  date  and  time 
of  the  incident,  hazardous  material  class  and  the  controlled  item  inventory 
code  designating  the  SRC  of  the  shipment,  a  description  of  the  shipment, 
the  name  of  the  carrier,  the  incident  location,  and  a  narrative  and  time- 
stamped  message  log  describing  the  incident. 


DOD’s  Use  of  DOT 
Safety  Performance 
Information  Does  Not 
Fully  Result  in 
Sufficient  and 
Reliable  Information 
to  Evaluate  the  Safety 
Performance  of 
Individual  TPS 
Carriers 


DOD’s  use  of  DOT  safety  performance  information  does  not  fully  result  in 
sufficient  and  reliable  information  to  evaluate  the  safety  performance  of 
individual  commercial  motor  carriers  in  the  TPS  program.  From 
November  2012  through  October  2014,  DOD  has  maintained  carriers  in 
the  TPS  program  that  have  absent  or  dated  Safety  Ratings  or  have 
exceeded  DOD’s  required  Safety  Measurement  System  (SMS)  score 
thresholds  in  at  least  one  of  the  Behavior  Analysis  and  Safety 
Improvement  Categories  (BASIC).  According  to  DOD  guidance, 
Satisfactory  Safety  Ratings  and  specific  Safety  Measurement  System 
score  thresholds  are  required  for  carriers  to  enter  and  be  retained  as 
carriers  entrusted  to  transport  security-sensitive  materials  in  the  TPS. 
DOD  has  partially  compensated  for  the  absent  or  dated  Safety  Ratings  by 
conducting  corporate  inspections — that  is,  inspections  to  identify  any 
gaps  in  TPS  carriers’  compliance  with  DOT  regulations  and  other  safety 
standards — of  some  carriers.  However,  DOD  has  not  compensated  for 
limitations  of  Safety  Measurement  System  scores,  which  we  found  also 
apply  to  TPS  carriers.  Further,  underpinning  Safety  Measurement  System 
scores  is  a  rich  source  of  publicly  available  violation  data  about  individual 
carrier’s  safety  performance  that  DOD  is  not  using  to  supplement  its 
safety  evaluation  of  individual  TPS  carriers. 
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DOD  Has  Maintained  TPS 
Carriers  That  Have  Absent 
or  Dated  Safety  Ratings, 
but  DOD’s  Corporate 
Inspections  Partially 
Address  These  Limitations 


Although  DOD  requires  that  carriers  have  Satisfactory  Safety  Ratings  to 
enter  and  remain  in  the  TPS  program,  we  found  that  DOD  had  maintained 
carriers  in  the  TPS  program  with  absent  Safety  Ratings.17  Specifically,  we 
found  that  of  the  55  carriers  that  DOD  had  contracted  with  to  transport 
hazardous  and  sensitive  materials  from  fiscal  year  201 1  through  fiscal 
year  2014,  7  carriers  did  not  have  a  Safety  Rating.  DOD  officials  reported 
that  the  7  carriers  without  Safety  Ratings  are  approved  to  handle  “general 
commodities”  (e.g.,  sensitive  documents)  for  DOD,  not  ammunition  or 
hazardous  materials  shipments.18  However,  DOD’s  guidance  does  not 
discuss  what  absent  Safety  Ratings  means  or  what  actions  DOD  staff  are 
to  take  when  such  ratings  are  missing.  Thus,  without  clear  guidance  on 
absent  safety  ratings,  it  will  be  difficult  for  DOD  to  provide  reasonable 
assurance  that  its  staff  are  consistently  and  effectively  evaluating  carriers 
entrusted  to  transport  security-sensitive  material  in  the  TPS  program 
consistent  with  its  requirements. 


We  also  found  that  DOD  maintained  carriers  in  the  TPS  program  that  had 
dated  Safety  Ratings.  DOD  does  not  have  a  requirement  for  the  age  of  a 
Safety  Rating.  Of  the  55  carriers  that  DOD  had  contracted  with  to 
transport  hazardous  and  sensitive  materials  from  fiscal  year  201 1  through 
fiscal  year  2014,  12  had  Safety  Ratings  that  were  20  or  more  years  old; 
23,  10  or  more  years  old;  and  38,  5  or  more  years  old.  Only  10  of  the  55 
TPS  carriers  had  ratings  that  were  fewer  than  5  years  old  (see  figure  1 ). 19 
Internal  controls  state  that  management  should  use  quality,  current 
information  from  relevant,  reliable,  and  timely  information,  and  that 
policies  and  procedures  help  ensure  that  management’s  directives  are 


17Military  Freight  Traffic  Unified  Rules  Publication-1  (MFTURP-1)  (June  3,  2013);  SDDC, 
Freight  Carrier  Registration  Program  Welcome  Package  (Aug.  2014).  The  MFTURP-1 
indicates  that  interstate  providers  must  maintain  a  satisfactory  safety  rating  with  FMCSA, 
while  intrastate  providers  must  maintain  a  satisfactory  safety  rating  with  the  applicable 
state  agency. 

1 ft 

Department  of  Defense  (DOD)  officials  also  reported  that  at  one  time,  it  was  not 
required  that  all  Transportation  Protective  Services  (TPS)  carriers  have  Satisfactory 
Safety  Ratings,  rather  that  TPS  carriers  not  have  an  Unsatisfactory  Safety  Rating.  DOD 
officials  could  not  tell  us  when  this  requirement  changed. 

1 Q 

One  Transportation  Protective  Service  (TPS)  carrier  had  a  Conditional  Safety  Rating  as 
of  June  2014;  however,  that  carrier  left  the  TPS  program  in  2013,  at  which  time  it  had  a 
Satisfactory  Safety  Rating.  For  the  purposes  of  this  graphic,  we  consider  this  carrier  to 
have  a  Satisfactory  Rating  that  is  fewer  than  10  years  old. 
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carried  out.20  Without  clear  guidance  on  how  DOD  is  to  address  carriers 
with  dated  Safety  Ratings,  the  information  they  provide  on  a  carrier’s 
safety  performance  may  be  limited. 


Figure  1:  Number  of  the  55  Transportation  Protective  Service  (TPS)  Carriers  DOD 
Contracted  with  from  Fiscal  Year  2011  through  Fiscal  Year  2014  with  Absent  or 
Dated  Safety  Ratings 


Safety  rating 
15 


10 


No 

rating 


Rating 
5  or  fewer 
years  old 


Rating 
5  to  10 
years  old 


Rating 
10  to  20 
years  old 


Rating 
20  or  more 
years  old 


Source:  GAO  analysis  of  Department  of  Transportation  (DOT)  data.  I  GAO-16-82 


TPS  carriers’  Safety  Ratings  could  also  be  absent  or  dated  because  DOT 
cannot  conduct  compliance  reviews  to  assign  Safety  Ratings  to  all 
500,000  carriers  operating  in  the  continental  United  States  in  a  given 
year.  For  example,  historically  FMCSA  and  its  state  partners  were  able  to 
conduct  compliance  reviews  on  about  3  percent  of  registered  commercial 
motor  carriers  annually,  meaning  that  the  vast  majority  of  carriers  are  not 
given  or  re-assigned  a  Safety  Rating  in  a  given  year.  DOT  prioritizes 
carriers  it  deems  “high-risk,”  as  described  above.  Once  a  carrier  is 
identified  as  high-risk,  according  to  DOT  it  is  then  further  prioritized  for 
investigation,  potentially  including  a  compliance  review,  based  on  factors 


20GAO,  Standards  for  Internal  Control  in  the  Federal  Government ,  GAO/AIM D-00-21 .3. 1 
(November  1999). 
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such  as  whether  the  carrier  was  high-risk  during  the  last  2  consecutive 
months,  or  whether  the  carrier  transports  hazardous  materials.  The 
Department  of  Transportation  (DOT)  has  been  planning  a  rulemaking  to 
change  the  way  it  assigns  Safety  Ratings  to  carriers  for  several  years. 
DOT  officials  estimated  DOT  would  issue  the  Notice  of  Proposed 
Rulemaking  in  October  2015.  Further,  GAO  was  directed  by  a  Senate 
Appropriations  Committee  report  to  monitor  the  implementation  of  CSA,21 
which  we  continue  to  do. 

DOD  guidance  on  requirements  for  carriers  indicates  that  TPS  carriers 
must  have  a  Satisfactory  Safety  Rating  with  FMCSA  to  enter  and  remain 
in  the  program,  and  that  DOD  may  conduct  unannounced  safety 
inspections  of  carriers’  facilities,  terminals,  equipment,  employees, 
records  and  procedures.22  However,  the  guidance  does  not  state 
consequences,  such  as  actions  its  staff  are  to  take,  when  carriers  do  not 
maintain  Satisfactory  Safety  Ratings.  For  comparison,  we  interviewed 
senior  officials  at  the  Department  of  Energy  Environmental  Management 
Program  because,  like  DOD,  the  Department  of  Energy  uses  commercial 
motor  carriers  to  transport  some  hazardous  materials.  In  contrast  to 
DOD’s  guidance,  the  Department  of  Energy’s  guidance  links  Safety 
Rating  requirements  with  the  specific  consequence  of  temporary  “non¬ 
use”  status  (i.e.  the  carrier  cannot  be  used).  Carriers  with  Safety  Ratings 
of  Conditional  or  Unsatisfactory  are  put  into  temporary  non-use  status, 
until  a  Satisfactory  Safety  Rating  is  restored.  In  contrast,  DOD  officials 
said  that  they  made  decisions  about  consequences  for  carriers  without 
Satisfactory  Safety  Ratings,  including  those  with  Conditional  Ratings,  in 
internal  discussions,  and  that  these  consequences  are  not  documented  in 
guidance.  For  example,  as  noted  above,  they  told  us  that  the  seven 
carriers  without  Safety  Ratings  are  not  approved  to  handle  ammunition  or 
hazardous  materials  shipments,  just  “general  commodities”  (e.g., 
sensitive  documents).  DOD  officials  told  us  that  they  could  not  remember 


21  See  S.  Rep.  No.  112-83,  at  52  (2011).  This  direction  is  contained  in  the  Senate 
Appropriations  Committee  Report  accompanying  the  Transportation  and  Housing  and 
Urban  Development,  and  Related  Agencies  Appropriations  Bill,  2012,  which  was 
eventually  included  in  the  Consolidated  and  Further  Continuing  Appropriations  Act,  2012, 
Pub.  L.  No.  112-55  (2011). 

22Military  Freight  T raffic  Unified  Rules  Publication-1  (MFTURP-1 )  (June  3,  2013);  see  also 
SDDC,  Freight  Carrier  Registration  Program  Welcome  Package  (Aug.  2014).  The 
MFTURP-1  indicates  that  interstate  providers  must  maintain  a  Satisfactory  Safety  Rating 
with  FMCSA,  while  intrastate  providers  must  maintain  a  Satisfactory  Safety  Rating  with 
the  applicable  state  agency. 
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an  instance  in  which  DOD  had  ever  placed  a  TPS  carrier  in  non-use 
status  due  to  absent  or  below  Satisfactory  Safety  Ratings.23  Standards  for 
Internal  Control  in  the  Federal  Government  state  that  internal  control 
activities,  such  as  policies  and  procedures,  help  ensure  that 
management’s  directives  are  carried  out.  Without  updating  guidance  on 
the  specific  actions  DOD  is  to  take  against  carriers  without  Satisfactory 
Safety  Ratings,  DOD  management  does  not  have  reasonable  assurance 
that  the  Department’s  Safety  Rating  requirements  are  being  carried  out. 

DOD  has  partially  compensated  for  the  absent  or  dated  Safety  Ratings  by 
conducting  “corporate  inspections” — that  is,  reviews  of  TPS  carriers  that 
cover  most  of  the  same  topics  covered  in  DOT’S  compliance  reviews.  To 
do  this,  DOD  uses  a  private-sector  company  to  identify  any  gaps  in  TPS 
carriers’  records  or  procedures  related  to  compliance  with  DOT 
regulations  on  hazardous  material  transport,  among  other  things. 
According  to  DOD  officials,  DOD  does  not  have  any  guidance  on  how 
often  to  conduct  corporate  inspections,  but  DOD  officials  told  us  that  they 
contract  with  a  private-sector  company  to  conduct  some  inspections 
every  year,  and  that  in  most  cases  large  TPS  carriers  are  inspected  more 
frequently  than  smaller  carriers.  DOD  officials  said  they  recently 
conducted  between  12  and  25  corporate  inspections  each  year.  Of  the  23 
carriers  that  we  had  identified  with  Safety  Ratings  older  than  10  years, 
DOD  told  us  that  19  of  them  had  received  and  passed  a  DOD  corporate 
inspection  in  the  last  4  years.24  DOD  told  us  that  of  the  7  carriers  we  had 
identified  without  Safety  Ratings,  all  had  received  and  passed  one 
corporate  inspection  since  2008. 


23We  identified  at  least  one  case,  in  2000  under  an  earlier  program  (and  subject  to  earlier 
guidance),  of  a  carrier  being  placed  in  non-use  status  by  the  Department  of  Defense  for 
failure  to  possess  a  Satisfactory  Department  of  Transportation  Safety  Rating.  See  Ready 
Transportation,  Inc.  v.  Military  Traffic  Management  Command,  86  F.  App’x  561,  563-64 
(4th  Cir.  2004). 

24Nineteen  TPS  carriers  received  Department  of  Defense  (DOD)  corporate  inspections 
from  201 1  through  2014.  DOD  officials  reported  that  three  did  not  because  they  were 
either  small  or  Canadian  companies.  DOD  officials  said  that  small  companies  are  not 
always  prioritized  for  corporate  inspections,  and  that  DOD  does  not  have  a  policy  on  how 
country-of-origin  affects  selection  for  corporate  inspections.  A  fourth  TPS  carrier  received 
a  more  focused  inspection  that  did  not  cover  all  of  the  areas  of  a  typical  corporate 
inspection. 


Page  16 


GAO-16-82  Defense  Transportation 


DOD  Has  Maintained  TPS 
Carriers  That  Have 
Exceeded  Required  Safety 
Measurement  System 
Score  Thresholds,  and 
Does  Not  Review 
Underlying  Violation  Data 
That  Could  at  Least 
Partially  Offset  the  Scores’ 
Limitations 


Although  DOD  requires  that  TPS  carriers  remain  below  certain  Safety 
Measurement  System  score  thresholds  in  the  seven  Behavior  Analysis 
and  Safety  Improvement  Categories  (BASIC),  we  found  that  from 
November  2012  through  October  2014,  DOD  had  maintained  carriers  in 
the  TPS  program  that  did  not  meet  those  requirements.25  For  example,  of 
the  53  TPS  carriers  with  data  in  our  2-year  data  set,26 12  exceeded  the 
Hours-of-Service  Compliance  threshold  in  1  or  more  months,  and  3 
exceeded  the  Hazardous  Materials  Compliance  threshold  in  1  or  more 
months.  DOD  senior  officials  reported  that  they  had  sent  letters  of 
concern  (letters  that  request  additional  information  on  these  scores)  to 
these  carriers.  In  table  2,  we  show  in  terms  of  1  month,  6  months,  and  12 
months,  the  extent  to  which  the  53  TPS  carriers  had  Safety  Measurement 
System  scores  that  were  above  the  required  threshold  in  each  category 
from  November  201 2  through  October  2014. 


Table  2:  Transportation  Protective  Services  (TPS)  Carriers  (53  Total)  with  Above-Threshold  Safety  Measurement  System 
Scores,  by  Category  (November  2012-October  2014) 


Unsafe 

Driving 

Hours-of- 

Service 

Compliance 

Driver 

Fitness 

Controlled 

Substances/ 

Alcohol 

Vehicle 

Maintenance 

Hazardous 

Materials 

Compliance 

Crash 

Indicator 

Carriers  above  threshold 
for  at  least  1  month 

11%  (6) 

23%  (12) 

6%  (3) 

0% 

11%  (6) 

6%  (3) 

9%  (5) 

Carriers  above  threshold 
for  more  than  6  months 

9%  (5) 

15%  (8) 

2%  (1) 

0% 

8%  (4) 

4%  (2  ) 

2%  (1 ) 

Carriers  above  threshold 
for  more  than  12  months 

6%  (3) 

11%  (6) 

0% 

0% 

6%  (3) 

2%  (1 ) 

2%  (1 ) 

Source:  GAO  Analysis  of  Department  of  Transportation  Data  |  GAO-16-82 


Note:  Safety  Measurement  System  scores  reflect  violations  across  a  carrier’s  entire  fleet,  which  may 
or  may  not  include  TPS  shipments. 


DOD  has  maintained  carriers  in  the  TPS  program  that  exceed  required 
Safety  Measurement  System  thresholds  in  part  because  DOD  guidance  is 
unclear  on  which  specific  trends  in  Safety  Measurement  System  scores 


25Military  Freight  Traffic  Unified  Rules  Publication-1  (MFTURP-1)  (June  3,  2013).  The 
guidance  specifies  that  carriers  providing  Dual  Driver  Protective  Service  and  Protective 
Security  Service  (two  TPS  services)  must  maintain  score  averages  of  60  or  lower  for  three 
categories  and  75  or  lower  for  the  remaining  four. 

26For  reasons  described  in  our  scope  and  methodology,  we  excluded  2  of  the  55  carriers 
in  the  Transportation  Protective  Service  program  from  fiscal  year  201 1  through  2014  that 
did  not  have  data  for  all  24  months  in  our  data  set. 
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might  prompt  which  specific  consequences.  For  entry  into  the  TPS 
program,  DOD  guidance  clearly  states  that  carriers  “with  scores  which 
exceed  the  established  DOT  threshold  levels  will  not  be  approved.”27 
However,  once  carriers  have  entered  the  TPS  program,  the  guidance  is 
less  clear  regarding  the  consequences  of  not  maintaining  the  required 
thresholds.  For  example,  the  guidance  states  that  SDDC  will  review  TPS 
carriers’  Safety  Measurement  System  scores  on  a  quarterly  basis,  and 
request  an  explanation  from  any  carrier  whose  score  is  above  the 
threshold  in  any  BASIC.  The  guidance  states  that  “[failure  to  provide  an 
adequate  explanation  or  to  show  improvement  may  result  in  the  [TPS 
carrier’s]  placement  into  non-use  status”  for  TPS  shipments.28  SDDC 
guidance  also  outlines  potential  consequences  of  exceeding  threshold 
scores,  such  as  sending  letters  of  concern  or  warning,  or  placement  in 
non-use  status.29  However,  the  guidance  does  not  further  define  what 
specific  Safety  Measurement  System  scores  or  trends,  such  as  scores 
above  the  threshold  for  a  certain  number  of  months  or  scores  above 
threshold  in  certain  BASIC  categories,  would  prompt  these  actions. 
Further,  senior  DOD  officials  said  that,  similar  to  Safety  Rating-related 
decisions,  decisions  about  consequences  for  carriers  that  did  not  meet 
Safety  Measurement  System  thresholds  were  made  based  on  internal 
discussion,  and  that  no  TPS  carrier  had  ever  been  placed  in  non-use 
status  for  Safety  Measurement  System  scores  that  were  above  the 
required  threshold,  or  for  any  other  indicator  of  poor  safety  performance. 
DOD  officials  said  they  use  Safety  Measurement  System  scores  as  one 
of  many  indicators  in  assessing  a  TPS  carrier’s  safety  and  determining 
what  additional  actions,  if  any,  are  needed.  For  example,  DOD  officials 
said  that  they  may  also  look  at  a  carrier’s  out  of  service  rates  for  drivers, 
vehicles,  or  vehicles  carrying  hazardous  materials.  However,  this  process 
was  described  as  ad  hoc  and  is  not  clearly  reflected  in  existing  guidance, 
meaning  that  DOD  has  not  fully  documented  how  TPS  carriers’  safety 
performance  is  evaluated. 


27SDDC,  Freight  Carrier  Registration  Program  Welcome  Package  (Aug.  2014  and  Sept. 
2013  versions).  As  stated  above,  we  were  unable  to  compare  TPS  carriers’  Safety 
Measurement  System  scores  at  entry  into  the  TPS  program  to  those  DOD  requires  of 
carriers  to  enter  the  TPS  program  because  the  CSA  Program — including  Safety 
Measurement  System  scores — was  not  implemented  until  2010,  and  most  TPS  carriers 
entered  the  program  before  then. 

28Military  Freight  Traffic  Unified  Rules  Publication-1  (MFTURP-1)  at  17  (June  3,  2013). 

29SDDC  Regulation  15-1,  Procedure  for  Disqualifying  and  Placing  Transportation  Service 
Providers  (TSP)  in  Non-Use,  app.  D,  paras.  2.w,  3.e(2)  (Jan.  10,  2012). 
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Standards  for  Internal  Control  In  the  Federal  Government  state  that 
internal  control  activities,  such  as  policies  and  procedures,  help  ensure 
that  management’s  directives  are  carried  out.30  However,  DOD’s  policies 
and  procedures  regarding  Safety  Measurement  System  score 
requirements  are  incomplete.  Thus,  they  may  be  inconsistently  applied, 
and  do  not  provide  reasonable  assurance  that  management’s  goals  for 
TPS  carrier  safety  performance  are  being  carried  out.  By  not  having  clear 
guidance  regarding  actions  DOD  should  take  against  TPS  carriers  that  do 
not  meet  Safety  Measurement  System  score  requirements,  DOD  does 
not  have  reasonable  assurance  that  only  TPS  carriers  that  reflect  desired 
standards  of  safety  performance  as  indicated  in  DOD’s  program 
requirements  are  transporting  security-sensitive  material. 

DOD  senior  officials  told  us  that  they  are  much  more  focused  on  TPS 
shipment-specific  data,  i.e.  the  driver  and  truck  transporting  the  TPS 
shipment  (which  comes  from  their  Defense  Transportation  Tracking 
System)  than  on  TPS  carrier-level  data,  such  as  Safety  Measurement 
System  scores.  While  it  is  reasonable  for  DOD  to  focus  on  individual  TPS 
shipments,  the  safety  record  of  the  carrier  across  all  of  its  drivers  and 
vehicles  provides  information  on  the  carrier’s  safety  policies  and  culture, 
which  can  impact  its  safety  performance.  The  Defense  Transportation 
Tracking  System  does  not  provide  this  information. 

In  February  2014,  as  mentioned  above,  we  reported  that  Safety 
Measurement  System  scores  for  carriers  with  fewer  than  20  vehicles  in 
their  fleet,  or  those  with  fewer  than  20  inspections,  are  less  reliable 
indicators  of  carriers’  relative  safety  performance  than  for  carriers  with 
more  vehicles  or  more  inspections,  which  may  limit  FMCSA’s  ability  to 
more  effectively  prioritize  carriers  for  intervention.31  During  the  course  of 


30GAO,  Standards  for  Internal  Control  in  the  Federal  Government,  GAO/AIM D-00-21 .3. 1 
(November  1999). 

31  In  our  February  2014  report  (see  GAO-14-1 14),  we  established  a  data  sufficiency 
standard  of  20  relevant  inspections  for  the  five  Behavior  Analysis  and  Safety  Improvement 
Categories  (BASIC)  that  use  inspections  to  calculate  a  Safety  Measurement  System 
score:  Hours-of-Service  Compliance,  Driver  Fitness,  Controlled  Substances/Alcohol, 
Vehicle  Maintenance,  and  Hazardous  Materials  Compliance.  FMCSA  often  uses  a  lower 
data  sufficiency  standard.  For  example,  in  the  Hours-of-Service  BASIC,  FMCSA  does  not 
calculate  Safety  Measurement  System  scores  for  a  carrier  unless  it  has  had  at  least  three 
inspections  and  at  least  one  violation  within  the  preceding  2  years.  We  found  that  by  only 
including  carriers  with  20  or  more  inspections,  we  were  able  to  identify  a  higher 
percentage  of  carriers  as  high-risk  that  subsequently  crashed  than  under  FMCSA’s  model. 
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this  review,  we  found  that  many  TPS  carriers  also  have  fewer  than  20 
vehicles  in  their  fleet  and  few  inspections,  and  therefore  the  limitations  we 
identified  in  February  2014  also  apply  to  some  TPS  carriers.  Of  the  53 
carriers  in  the  TPS  program  from  fiscal  year  201 1  through  fiscal  year 
2014  for  which  we  had  24  months  of  data,  20  carriers  (38  percent)  had 
fewer  than  20  vehicles,  and  many  of  the  53  had  fewer  than  20 
inspections.32  For  example,  in  figure  2  we  show  that  34  percent  of  TPS 
carriers  had  fewer  than  20  inspections  in  the  Hazardous  Materials 
Compliance  category.  Given  that  TPS  carriers  often  transport  hazardous 
materials,  inspections  in  that  category  are  particularly  relevant  for  DOD. 


Figure  2:  Percentage  of  Transportation  Protective  Services  (TPS)  Carriers  (of  53)  with  Fewer  Than  20  Inspections  by  Behavior 
Analysis  and  Safety  Improvement  Category  (BASIC)  (November  2012  -  October  2014) 


Vehicle  Maintenance 


Hazardous  Materials  Compliance 


Controlled  Substances/Alcohol 


Driver  Fitness 


Hours-of-Service  Compliance 


Unsafe  Driving 


Crash  Indicator 


0  10  20  30  40  50  60  70  80  90  100 

Percent 

Source:  GAO  analysis  of  Department  of  Transportation  (DOT)  data.  I  GAO-16-82 

Notes:  Safety  Measurement  System  scores  in  the  Unsafe  Driving  and  Crash  Indicator  BASICS  are 
calculated  using  the  carrier’s  number  of  vehicles  rather  than  the  number  of  inspections,  so  the 
number  of  inspections  is  less  relevant. 


32 As  stated  above,  two  of  the  55  TPS  carriers  DOD  contracted  with  from  fiscal  year  201 1 
through  fiscal  year  2014  did  not  have  Safety  Measurement  System  scores  for  all  24 
months  in  our  data  set  as  they  stopped  being  active  carriers,  and  thus  the  Department  of 
Transportation  stopped  assigning  those  scores,  before  the  end  of  October  2014,  the  last 
month  in  our  data  set. 
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In  our  February  2014  report,  we  noted  that  most  carriers  in  the  United 
States  did  not  have  sufficient  information  to  receive  Safety  Measurement 
System  scores  in  each  of  the  seven  Behavior  Analysis  and  Safety 
Improvement  Categories.  The  same  is  true  of  TPS  carriers.  In  figure  3  we 
show,  for  example,  that  many  of  the  53  TPS  carriers  did  not  receive 
scores  every  month  in  some  categories  during  the  November  2012 
through  October  2014  time  frame.  For  example,  77  percent  of  the  53 
carriers  did  not  receive  a  score  in  the  Hazardous  Materials  Compliance 
category  in  all  24  months. 


Figure  3:  Transportation  Protective  Services  (TPS)  Carriers  (of  53)  That  Did  Not  Receive  Safety  Measurement  System  Scores 
in  Specific  Behavior  Analysis  and  Safety  Improvement  Categories,  November  2012  -  October  2014 
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Underlying  Violation  Data 
Are  a  Rich  Source  of 
Information 


Though  Safety  Measurement  System  scores  have  limitations,  we  found 
that  DOD  is  not  analyzing  trends  in  violation  data  used  to  calculate  these 
scores,  which  are  a  potentially  rich  source  of  information  on  carriers’  on- 
the-road  safety  performance,  because  DOD  does  not  require  its  staff  to 
review  these  violation  data.  These  violation  data  provide  additional  detail 
that  the  Safety  Measurement  System  scores  do  not,  and  are  a  more 
reliable  source  of  information.  For  example,  in  2014,  the  DOT  Inspector 
General  found  that  FMCSA  had  controls  in  place  to  improve  the  quality  of 
state-reported  data,  which  includes  violations.  FMCSA  also  has  a  process 
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through  which  carriers  and  others  can  request  corrections  to  state- 
reported  data  if  they  identify  inaccuracies.  Further,  violations  data  are 
readily  available  from  FMCSA’s  publicly  available  website.  To  illustrate,  in 
figures  4,  5,  and  6,  we  show  that  by  clicking  on  the  publicly  available 
website  where  FMCSA  reports  its  Safety  Rating  and  Safety  Measurement 
System  scores,  DOD  could  easily  and  quickly  access  the  violation  data 
related  to  the  Unsafe  Driving  BASIC.  Clicking  on  that  category  in  the 
screenshot  displayed  in  figure  4  would  lead  DOD  to  the  Unsafe  Driving 
BASIC  score  displayed  in  figure  5.  And  clicking  on  that  score  would  lead 
DOD  to  a  summary  of  the  violation  data  displayed  in  figure  6.  The 
summary  includes  violations  such  as  speeding  and  using  a  cell  phone 
while  driving,  among  others. 
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Figure  4:  FMCSA’s  Publicly  Available  Website  Showing  Summary  Information  on  the  Seven  Behavior  Analysis  and  Safety 
Improvement  Categories  (BASICs) 
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Figure  5:  FMCSA’s  Publicly  Available  Website  Showing  Information  on  the  Unsafe 
Driving  Behavior  Analysis  and  Safety  Improvement  Category  (BASIC) 
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Source:  Federal  Motor  Carrier  Safety  Administration  (FMCSA).  |  GAO-16-82 
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Figure  6:  FMCSA’s  Publicly  Available  Website  Showing  Underpinning  Violation  Data  on  the  Unsafe  Driving  Behavior  Analysis 
and  Safety  Improvement  Category  (BASIC) 
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Source:  Federal  Motor  Carrier  Safety  Administration  (FMCSA).  |  GAO-16-82 


Violations  do  not  necessarily  suggest  that  a  carrier  is  unsafe — the  more 
vehicles  a  carrier  has,  the  more  likely  it  is  to  incur  violations — but  violation 
data  are  sources  of  information  that  could  be  used  to  analyze  meaningful 
trends  within  and  across  carriers,  particularly  for  carriers  that  do  not  have 
Safety  Measurement  System  scores.  In  figure  5,  the  carrier  has  a  score  of 
79  in  the  Unsafe  Driving  category,  which  is  above  the  DOT-designated 
threshold  of  65  for  general  carriers.  The  underpinning  list  of  violations  in 
figure  6  provides  more  detailed  information  on  that  score.  Even  if  a  carrier 
does  not  have  a  score  in  a  particular  category,  the  underpinning  violations 
records  are  available,  thus  providing  information  where  a  Safety 
Measurement  System  score  does  not.  For  example,  analyzing  FMCSA’s 
violation  data,  we  found  that  all  but  one  of  the  53  TPS  carriers  was  listed 
as  having  violations  in  at  least  one  category.  For  instance,  in  figure  7,  we 
show  that  6  TPS  carriers  had  Controlled  Substances/Alcohol  violations,  a 
category  that  includes  those  incurred  for  using  or  possessing  drugs  while 
operating  a  commercial  motor  vehicle,  using  alcohol  within  a  certain 
period  of  time  prior  to  duty,  and  violating  a  related  order  to  remain  out-of- 
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service.  In  addition,  19  of  the  53  TPS  carriers  incurred  violations  in  the 
Hazardous  Materials  Compliance  category.  Additionally,  while 
transporting  hazardous  materials,  20  of  the  53  TPS  carriers  incurred 
Unsafe  Driving  violations  and  22  carriers  incurred  Vehicle  Maintenance 
violations,  according  to  DOT  safety  performance  data.  It  is  important  to 
note  that,  though  these  violations  occurred  among  TPS  carriers  while 
transporting  hazardous  materials,  we  do  not  know  whether  it  was  while 
transporting  shipments  for  DOD  or  for  another  entity.  Since  a  TPS  carrier 
may  transport  for  DOD  and  other  agencies  or  companies,  some  of  its 
drivers  and  trucks  transport  TPS  shipments  and  others  do  not. 


Figure  7:  Transportation  Protective  Services  (TPS)  Carriers  with  Violations  Related 
to  Each  Behavior  Analysis  and  Safety  Improvement  Category  (BASIC),  Including 
Those  Incurred  While  Transporting  Hazardous  Materials  (November  2012-October 
2014) 
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Page  26 


GAO-16-82  Defense  Transportation 


Consistent  with  DOD’s  guidance,  DOD  officials  told  us  they  review  Safety 
Measurement  System  scores  on  a  quarterly  basis,  and  use  them  as  one 
of  many  indicators.33  They  do  not  examine  underlying  violation  data, 
including  for  trends  across  or  within  carriers.  Therefore,  they  were  not 
aware  of  the  Controlled  Substances/Alcohol  and  Hazardous  Materials 
Compliance  violations  data  we  characterized  in  our  example  and  did  not 
take  any  mitigating  action.  However,  one  senior  DOD  official  told  us  that  it 
would  be  possible  to  use  the  violation  data  to  identify  specific  TPS 
shipments  by  matching  the  vehicle  identification  numbers  (VIN)  and  dates 
of  TPS  shipments,  to  those  in  DOT’S  violation  data.  This  approach  would 
provide  DOD  with  violation  data  specific  to  TPS  shipments  and  by 
extension  their  drivers  and  trucks,  but  DOD  is  not  currently  conducting 
this  analysis.  DOD  officials  told  us  they  do  not  have  the  staff  that  would 
be  necessary  to  review  underpinning  violation  data,  and  that  regardless 
as  with  the  case  with  compliance  reviews  and  corporate  inspections,  they 
are  more  focused  on  TPS  shipment-specific  data,  coming  from  the 
Defense  Transportation  Tracking  System,  than  on  carrier-level  data  from 
DOT.  The  Defense  Transportation  Tracking  System,  described  above, 
records  whether  each  shipment  reaches  its  destination,  and  does  not 
capture  carrier-level  safety  performance  data,  which  is  important  because 
it  can  reflect  a  carrier’s  safety  policies  and  culture.  Further,  Standards  for 
Internal  Control  in  the  Federal  Government  state  that  “management 
should  use  quality,  current  information  from  relevant,  reliable,  and  timely 
information,  and  that  policies  and  procedures  help  ensure  that 
management’s  directives  are  carried  out.”34  Reviewing  the  available 
violation  data  could  enable  DOD  to  more  fully  evaluate  the  safety 
performance  of  individual  carriers  entrusted  to  transport  potentially 
dangerous  or  hazardous  materials. 


oo 

DOT’S  Compliance,  Safety,  Accountability  website  states  that  “readers  should  not  draw 
conclusions  about  a  carrier’s  overall  safety  condition  simply  based  on  the  data  displayed 
in  [the  SMS]  system.” 

34GAO,  Standards  for  Internal  Control  in  the  Federal  Government,  GAO/AIM D-00-21 .3. 1 
(November  1999). 
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DOD  Collects  TPS 
Carrier  Incident  Data 
but  Does  Not 
Evaluate  the  Data  or 
Fully  Investigate  TPS 
Carrier  Incidents  to 
Identify  Safety- 
Related  Trends 


DOD  collects  incident  data  involving  TPS  carriers  of  security-sensitive 
materials  in  its  Defense  Transportation  Tracking  System  for  emergency 
response  and  generates  an  incident  report;  however,  the  department 
does  not  evaluate  TPS  carrier  incident  data  to  determine  whether 
systemic  trends  and  patterns  are  linked  to  safety  risks.  Further,  DOD 
does  not  fully  investigate  incidents  of  TPS  carriers  transporting  security- 
sensitive  materials  to  determine  the  root  causes  of  the  incidents,  which 
further  prevents  the  identification  of  safety-related  trends. 


DOD  Uses  the  Defense 
Transportation  Tracking 
System  to  Collect  Incident 
Data  but  Does  Not 
Evaluate  Them  to 
Determine  Whether  They 
Are  Indicative  of  Systemic 
Trends 


DOD  uses  the  Defense  Transportation  Tracking  System  to  collect  incident 
data  on  TPS  carriers  to  respond  to  emergencies  involving  the  transport  of 
security-sensitive  materials.  However,  the  department  is  not  evaluating 
incident  data  to  determine  whether  systemic  trends  and  patterns  exist  that 
are  indicative  of  safety  risks  for  TPS  carriers  transporting  security- 
sensitive  materials.  DOD  Instruction  6055.16,  Explosives  Safety 
Management  Program ,  which  provides  an  overview  of  requirements  and 
procedures  for  explosives  safety  management,  requires  the  military 
departments  and  defense  agencies  to  establish  explosives  safety 
management  programs  and  describes,  among  other  things,  the 
importance  of  conducting  appropriate  analyses  of  incidents  involving 
DOD  ammunitions  and  explosives  to  reduce  the  impact  of  and  to  prevent 
future  mishaps.35  According  to  this  instruction,  careful  analyses  can 
identify  appropriate  risk  mitigation  actions,  which  in  the  case  of  TPS  could 
better  protect  the  drivers,  the  shipment,  and  the  general  public.  DOD 
Instruction  6055.07,  Mishap  Notification,  Investigation,  Reporting,  and 
Record  Keeping,  which  contains  more  detailed  guidance  related  to  the 
evaluation  and  investigations  of  DOD  incidents,  generally  describes  a 
DOD  mishap  as  an  unplanned  event  or  series  of  events  resulting  in, 


35  See  Department  of  Defense  Instruction  6055.16,  Explosives  Safety  Management 
Program,  end.  2,  paras.  4-5  (July  29,  2008)  (incorporating  change  Dec.  8,  201 1 );  id. 
end.  12. 
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among  other  things,  damage  to  property,  illness,  or  injury  caused  by  DOD 
activities.36 

The  Defense  Transportation  Tracking  System,  which  is  managed  by 
SDDC,  contains  comprehensive  information  related  to  TPS  carrier 
incidents.  Our  examination  of  the  Defense  Transportation  Tracking 
System  incident  data  found  potential  areas  where  systemic  trends  or 
patterns  related  to  safety  might  exist.  For  example,  on  the  basis  of  our 
review  of  the  1 ,039  incident  reports  from  the  Defense  Transportation 
Tracking  System  from  fiscal  years  201 1  through  2014,  we  found  that  the 
72  percent  or  749  incident  reports  involving  TPS  carriers  were  related  to 
mechanical  breakdowns  (see  figure  8). 


Figure  8:  TPS  Carrier  Incident  Data  in  the  Defense  Transportation  Tracking  System 
by  Category  (Fiscal  Years  2011  through  2014) 


Source:  GAO  analysis  of  Defense  Transportation  Tracking  System  (DTTS)  data.  |  GAO-16-82 


36  Specifically,  Department  of  Defense  (DOD)  Instruction  6055.07  defines  a  DOD  mishap 
as  an  unplanned  event  or  series  of  events  that  results  in  damage  to  DOD  property; 
occupational  illness  to  DOD  personnel;  injury  to  DOD  personnel  (on-  or  off-duty  military 
personnel  and  on-duty  civilian  personnel);  or  damage  to  public  or  private  property,  or 
injury  or  illness  to  non-DOD  personnel,  caused  by  DOD  activities.  See  Department  of 
Defense  Instruction  6055.07,  Mishap  Notification,  Investigation,  Reporting,  and  Record 
Keeping  at  47  (June  6,  201 1 ). 
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Further,  our  analysis  shows  that  there  is  an  inverse  relationship  between 
the  numbers  of  mechanical  breakdowns  and  shipments  involving  TPS 
carriers.  The  number  of  mechanical  breakdowns  increased  from  132 
incidents  in  fiscal  year  201 1  to  314  incidents  in  fiscal  year  2014,  while  the 
number  of  shipments  decreased  from  65,198  in  fiscal  year  201 1  to  46,036 
in  fiscal  year  2014  (see  figure  9). 


Figure  9:  Inverse  Relationship  between  the  Number  of  Mechanical  Breakdowns  and 
the  Number  of  Shipments  (Fiscal  Year  201 1  Compared  with  Fiscal  Year  2014) 
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These  mechanical  breakdown  incidents  have  resulted  in  delays  that, 
according  to  SDDC  officials  increased  the  exposure  of  security-sensitive 
material  to  the  general  public.  Based  on  our  analysis  of  the  749 
mechanical  breakdown  incidents,  we  found  that  98  percent  of  the  TPS 
carriers  that  had  mechanical  breakdowns  while  transporting  the  highest- 
risk  materials  (e.g.,  Security  Risk  Category  i  and  II  materials  such  as 
stinger  missiles  and  grenades)  stopped  for  more  than  2  hours.  Similarly, 
we  found  that  90  percent  of  the  TPS  carriers  that  had  mechanical 
breakdowns  while  transporting  SRC  III  and  IV  materials  such  as  bulk 
explosives  and  fuses  stopped  for  more  than  4  hours. 
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If  DOD  were  systematically  analyzing  incident  data  related  to  mechanical 
breakdowns,  DOD  could  have  determined  that  some  of  these  mechanical 
breakdown  incidents  could  have  been  detected  through  DOD’s  motor 
vehicle  inspection  process.  Chapter  205  of  the  Defense  Transportation 
Regulation  requires  DOD  transportation  officers  to  conduct  inspections  of 
vehicles  used  to  transport  certain  security-sensitive  materials  before 
loading  shipments,  to  ensure  vehicles  are  in  satisfactory  condition,  using 
DOD’s  Motor  Vehicle  Inspection  form.37  During  the  inspection,  DOD 
transportation  officers  are  to  inspect  vehicle  equipment  such  as  the 
electrical  wiring  system,  the  satellite  motor  surveillance  system,  the 
steering  system,  the  warning  equipment,  the  fuel  and  exhaust  systems, 
the  brakes,  and  the  cargo  space  for  the  DOD  materials.  According  to 
DOD’s  Motor  Vehicle  Inspection  form ,  unsatisfactory  equipment 
conditions  identified  during  the  inspection  must  be  corrected  before  the 
security-sensitive  materials  are  loaded.  SDDC  officials  told  us  that  all 
vehicles  transporting  DOD  security-sensitive  materials,  including  the  749 
mechanical  breakdown  incidents  in  our  review,  should  have  been 
inspected  to  detect  and  correct  deficiencies  prior  to  loading.  In  examining 
incident  data  from  the  Defense  Transportation  Tracking  System,  we  found 
that  while  some  mechanical  breakdown  incidents  could  not  have  been 
avoided,  other  mechanical  breakdowns  incidents  might  have  been 
avoided  had  the  issue  been  detected  in  the  inspection  prior  to  loading. 

For  example: 

•  In  May  2014,  a  TPS  carrier’s  truck  broke  down  while  delivering 
explosives  due  to  problems  with  its  electrical  system.  According  to 
DOD’s  Motor  Vehicle  Inspection  form,  electrical  system  wires  should 
be  examined  during  the  inspection  and  any  deficiencies  should  be 
corrected  before  loading  the  shipments.  While  the  TPS  carrier’s  truck 
was  repaired,  another  TPS  truck  was  used  to  return  the  shipment  to 
the  point  of  origin.  A  third  TPS  truck  attempted  to  deliver  the  shipment 
to  the  destination  but  also  experienced  mechanical  failure.  A  fourth 
TPS  truck  delivered  the  shipment  to  its  final  destination,  delaying  the 
transport  by  over  6  days. 


37  Department  of  Defense,  Defense  Transportation  Regulation  4500. 9-R,  pt.  II,  ch.  205, 
Transportation  Protective  Service  (TPS)  (July  27,  2015).  Chapter  205  requires  use  of  the 
form  for  arms,  ammunition,  and  explosives  shipments  requiring  certain  TPS  services,  as 
well  as  hazardous  materials  shipments. 
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•  In  May  2012,  a  truck  driver  carrying  17,280  hand  grenades  made  an 
unscheduled  stop  to  repair  the  satellite  equipment.  It  was  later 
revealed  and  reported  that  the  DOD  transportation  officer  that  had 
inspected  the  truck  knew  that  the  satellite  equipment  was  not  working 
before  the  driver  left  the  loading  dock.  For  arms,  ammunition,  and 
explosives  and  other  security  sensitive  shipments  requiring  satellite 
equipment,  DOD’s  Motor  Vehicle  Inspection  form  requires  the  DOD 
inspector  to  ensure  that  the  satellite  equipment  is  operable  as  part  of 
the  inspection  prior  to  loading. 

•  In  July  2013,  a  loose  bolt  on  a  TPS  carrier’s  trailer  was  identified 
during  a  DOD  roadside  inspection  following  DOD’s  motor  vehicle 
inspection.  The  carrier  was  transporting  ammunition  cartridges.  The 
trailer,  including  components  such  as  bolts,  should  have  been 
inspected  during  DOD’s  vehicle  inspection  process  before  the  driver 
left  the  loading  dock.  The  DOT  inspectors  required  the  driver  to  repair 
the  loose  bolt  before  leaving  the  DOT  inspection  facility  and  delivering 
the  shipment. 

In  other  cases,  if  DOD  had  systematically  analyzed  TPS  carrier  incident 
data  related  to  mechanical  breakdowns  and  the  carrier  had  conducted 
preventive  maintenance  on  vehicles,  delays  resulting  from  TPS  carriers 
stopping  at  commercial  repair  shops  for  scheduled  maintenance  may 
have  been  avoided.  We  found  that  70  percent  of  the  749  mechanical 
breakdown  incidents  involved  drivers  that  had  stopped  at  a  commercial 
repair  facility  while  delivering  security-sensitive  cargo.  However, 
according  to  DOD  Explosives  Safety  Board38  and  Army  Safety  officials, 
some  of  this  unscheduled  maintenance  may  have  been  avoided  or,  at  the 
least,  mitigated  if  the  carrier  had  conducted  adequate  routine  and 
preventive  maintenance  on  vehicles.39  Although  we  do  not  know  the 


38  The  purpose  of  the  Department  of  Defense  (DOD)  Explosives  Safety  Board  is  to  protect 
the  public,  military  personnel,  and  government  and  private  property  from  unintentional 
consequences  of  an  incident  or  accident  (e.g.,  fire  and  explosion)  involving  military 
munitions.  The  Board  recommends  policy,  develops  and  provides  technical  guidance, 
evaluates  programs,  and  provides  a  forum  for  coordination  among  the  DOD  components 
on  all  matters  related  to  explosives  safety  management. 

oq 

Officials  noted  that,  under  Department  of  Transportation  regulations,  drivers  are  to  be 
satisfied  that  their  motor  vehicles  are  in  safe  operating  condition  before  driving.  See  49 
C.F.R.  §  396.13.  Motor  carriers  also  must  systematically  inspect,  repair,  and  maintain  their 
vehicles  and  ensure  that  parts  and  accessories  are  in  safe  and  proper  operating  condition, 
see  §  396.3,  as  well  as  require  their  drivers  to  report  on  defects  or  deficiencies  that  would 
affect  the  safety  of  operation  or  result  in  mechanical  breakdown.  See  §  396.1 1 . 
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extent  to  which  DOD  carriers  had  conducted  adequate  and  preventive 
maintenance  on  vehicles,  the  following  examples  illustrate  mechanical 
breakdown  incidents  reported  through  the  Defense  Transportation 
Tracking  System  that  resulted  in  the  driver  stopping  at  a  commercial 
repair  facility  for  maintenance  and  repair  while  delivering  security- 
sensitive  materials. 

•  In  February  2012,  a  TPS  carrier’s  driver  contacted  personnel  from  the 
Defense  Transportation  Tracking  System  office  to  inform  staff  of 
electrical  issues  and  that  they  had  stopped  at  a  commercial  repair 
facility  for  repair  due  to  mechanical  failure  while  delivering  rockets. 

The  TPS  carrier’s  driver  departed  from  the  commercial  repair  facility 
after  the  repairs  had  been  completed,  but  the  truck  subsequently 
broke  down  a  second  time.  Although  the  TPS  carrier’s  driver  returned 
the  truck  to  the  commercial  repair  facility  to  resolve  the  same  issue, 
the  truck  broke  down  a  third  time.  As  a  result  of  these  recurring 
mechanical  issues,  the  cargo  had  to  be  transferred  to  another  truck 
for  delivery  and  the  shipment  was  delayed  by  nearly  3  days,  which 
according  to  Army  Safety  officials  may  have  been  avoided  through 
proper  routine  maintenance  of  the  electrical  system. 

•  In  December  2012,  a  TPS  carrier’s  driver  transporting  release  devices 
for  launching  explosives  noticed  an  oil  leak  from  the  truck  that  was 
later  found  to  be  the  result  of  a  blown  oil  pressure  line.  The  driver 
stopped  at  a  commercial  repair  facility  for  repairs.  However,  it  took 
nearly  4  days  to  repair  the  blown  oil  pressure  line  and  depart  for  the 
shipping  destination.  According  to  Army  Safety  officials  inspection  and 
replacement  of  oil  pressure  lines  are  generally  included  as  part  of 
routine  preventative  maintenance  and  so  this  breakdown  could 
potentially  have  been  avoided. 

•  In  February  2014,  a  TPS  carrier’s  driver  delivering  security-sensitive 
missile  guidance  and  control  systems  had  problems  with  the  truck’s 
fuel  filter,  which  is  typically  serviced  as  part  of  the  preventative 
maintenance  process.  A  service  truck  attempted  to  repair  the  fuel 
filter,  but  the  original  truck  ultimately  broke  down.  The  service  truck 
returned  to  perform  the  repairs  again,  but  was  unsuccessful.  The  truck 
was  later  towed  to  a  commercial  maintenance  facility  and  was 
successfully  repaired.  However,  due  to  the  mechanical  issues,  the 
delivery  was  delayed  by  10  hours.  According  to  Army  officials, 
inspection  and  replacement  of  the  fuel  filter  is  generally  considered 
part  of  routine  maintenance. 
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In  addition,  SDDC  officials  said  they  believe  that  in  some  cases  TPS 
carriers  have  established  a  pattern  of  stopping  at  commercial 
maintenance  shops  for  routine  and  preventive  maintenance  while 
delivering  security-sensitive  materials.  The  Defense  Transportation 
Tracking  System  showed  and  TPS  contracting  office  officials  told  us  that 
some  carriers  regularly  go  to  commercial  repair  facilities  for  routine 
maintenance  and  repairs  while  delivering  DOD  shipments  of  security- 
sensitive  materials,  but  DOD  does  not  know  the  magnitude  of  such 
occurrences  and  therefore  cannot  resolve  the  situation  because  it  does 
not  evaluate  incident  data  for  systemic  patterns  or  trends.  DOD  Directive 
6055. 9E,  Explosives  Safety  Management  and  the  DOD  Explosives  Safety 
Board,  indicates  that  it  is  DOD  policy  to  minimize  the  amount  of  time  and 
number  of  people  exposed  to  explosives.40  However,  in  reviewing 
information  in  the  Defense  Transportation  Tracking  System,  we  found 
instances  where  TPS  carriers  transporting  security-sensitive  materials 
experienced  significant  delays  when  they  stopped  at  commercial  repair 
facilities.  For  example,  in  April  2012,  a  TPS  carrier  transporting  devices 
for  releasing  explosives  stopped  at  a  commercial  repair  facility  for  routine 
maintenance  on  the  truck  and  brake  work  on  the  trailer,  resulting  in  a 
delay  of  9  hours.  In  another  example,  in  January  2014,  a  TPS  carrier 
transporting  guided  missiles  stopped  at  a  commercial  repair  facility  to  get 
the  air  conditioning  system  on  the  truck  repaired,  delaying  the  delivery  by 
13  hours  and  increasing  exposure  of  these  sensitive  materials  to  the 
public. 

DOD  does  not  evaluate  TPS  carrier  incident  data  from  the  Defense 
Transportation  Tracking  System  because  its  existing  guidance  does  not 
require  the  evaluation  of  this  data  to  identify  trends  and  patterns  that 
could  suggest  systemic  weaknesses  such  as  frequent  mechanical 
breakdowns  or  unusual  delays.  Standards  for  Internal  Control  in  the 
Federal  Government  state  that  management  should  establish  monitoring 
activities  and  evaluate  the  results.  Analyzing  Defense  Transportation 
Tracking  System  data  could  help  identify  patterns  and  trends  in 
mechanical  breakdown  incidents  involving  TPS  carriers  and  identify 
systemic  issues  that  could  be  addressed  through  corrective  action.  For 
example,  an  analysis  of  these  incidents  could  result  in  the  identification  of 
deficiencies  in  the  vehicle  inspection  process  or  the  need  for  more 


40  See  Department  of  Defense  Directive  6055. 9E,  Explosives  Safety  Management  and  the 
DOD  Explosives  Safety  Board,  para.  4.4  (Aug.  19,  2005). 
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rigorous  enforcement  of  preventative  maintenance  requirements, 
performed  prior  to  loading  security-sensitive  materials.  DOD  officials  at 
the  office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology 
and  Logistics,  Transportation  Policy  have  acknowledged  weaknesses  in 
the  vehicle  inspection  process  and  have  taken  initial  steps  in  determining 
how  to  improve  the  motor  vehicle  inspection  program.  However,  DOD 
does  not  require  an  evaluation  of  TPS  carrier  data  from  the  Defense 
Transportation  Tracking  System  to  identify  systemic  trends  and  patterns 
that  might  be  linked  to  safety  risks.  DOD  Explosives  Safety  Board  and 
Army  Safety  officials  stated  that  requiring  the  evaluation  of  incident  data 
may  help  DOD  improve  the  safe  transport  of  security-sensitive  materials. 
Although  the  Defense  Transportation  Tracking  System  was  developed  to 
respond  to  emergencies,  it  can  also  track  mechanical  incidents  that  can 
be  used  for  evaluating  and  analyzing  incidents.  Without  requiring  the 
evaluation  of  TPS  carrier  incident  data  to  determine  whether  there  are 
systemic  issues  that  have  led  to  the  incidents  reported,  DOD  is  not  in  the 
best  position  to  mitigate  future  recurrences  of  incidents  such  as 
mechanical  breakdowns. 


DOD  Does  Not  Fully 
Investigate  Incidents 
Regarding  TPS  Carriers  to 
Identify  Their  Root  Causes 


In  addition  to  not  evaluating  trends  in  incident  data,  DOD  also  does  not 
fully  investigate  incidents  regarding  TPS  carriers  to  determine  their  root 
causes  to  identify  trends  in  safety-related  risks.  In  other  contexts,  DOD 
guidance  provides  for  notification  and  investigation  of  mishaps. 
Specifically,  DOD  Instruction  6055.07,  Mishap  Notification,  Investigation, 
Reporting  and  Record  Keeping,  provides  guidance  for  the  investigation  of 
various  categories  of  mishaps.  For  example,  with  respect  to  fire  loss 
investigations,  the  guidance  requires  the  investigation  to  identify  the 
cause  of  the  fire.41  Further,  Army  Pamphlet  385-40,  Army  Accident 
Investigations  and  Reporting,  provides  guidance  on  conducting  an 
investigation,  noting  that  the  procedures  it  provides  are  designed  to  assist 
the  investigator  in  identifying  the  key  factors  that  caused  or  contributed  to 
the  accident  as  well  as  the  root  cause.  Through  the  investigation, 
recommendations  are  identified  that  will  remedy  the  causes  and  minimize 
the  chances  for  similar  recurrences.42  Although  DOD  Instruction  6055.07 


41 

See  Department  of  Defense  Instruction  6055.07,  Mishap  Notification ,  Investigation , 
Reporting ,  and  Record  Keeping,  end.  4,  para.  2  (June  6,  201 1 ). 

42  See  Army  Pamphlet  385-40,  Army  Accident  Investigations  and  Reporting,  paras.  1-4,  1- 
5  (Mar.  18,  2015). 
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covers  the  investigation  and  reporting  of  mishaps  resulting  from 
contractor  operations  that  involve  damage  to  DOD  property  or  injury  to 
DOD  personnel,  a  list  of  exemptions  specifically  includes  contractor 
mishaps  in  which  the  contractor  employee  is  not  under  the  direct 
supervision  of  DOD  personnel.43  According  to  Army  Safety  officials,  this 
would  exempt  circumstances  involving  TPS  shipments.  Incidents  such  as 
an  accident  involving  TPS  carriers  delivering  explosives  can  be 
catastrophic  if  the  explosives  are  ignited. 

On  the  basis  of  available  documentation,  we  found  that  three  of  the  four 
investigations  conducted  by  the  SDDC  since  2012  were  preliminary  and 
incomplete.  For  example,  DOD  did  not  identify  the  root  cause  for  these 
three  incidents,  which  limits  DOD  in  precluding  similar  incidents  from 
recurring  and  from  identifying  any  safety-related  trends. 

Specifically,  we  found: 

•  In  April  2014,  Army  Safety  personnel  began  a  preliminary 

investigation  of  an  incident  near  Vian,  Oklahoma,  that  occurred  during 
the  same  month  involving  a  TPS  carrier’s  vehicle  trailer  that  caught  on 
fire  (see  fig.  10).  There  were  no  reported  injuries  or  documented 
damage  to  the  explosives,  but  this  incident  could  have  been  more 
serious  had  the  fire  detonated  the  explosives  in  the  trailer  or  cargo 
area.  The  preliminary  investigation  of  the  TPS  carrier  was  never 
completed  because  the  root  cause  of  the  incident  was  never 
determined.  According  to  Army  Safety  officials,  it  was  never 
determined  whether  the  fire  was  caused  by  leaking  brake  fluid  or  by 
another  preventable  mechanical  issue  with  the  trailer.  Without 
knowing  the  root  cause,  contributory  factors  and  safety  risks,  DOD  is 
not  positioned  to  take  action  to  prevent  a  future  recurrence.  In  figure 
10,  we  provide  photographs  of  the  underside  of  the  TPS  carrier’s 
trailer  that  caught  on  fire  near  Vian,  Oklahoma,  in  April  2014  and  of 
the  hundreds  of  M795  155mm  high  explosive  projectiles  that  were 
inside  the  trailer. 


43  Army  Regulation  385-10,  which  is  implemented  by  the  investigation  procedures  in  Army 
Pamphlet  385-40,  similarly  specifies  that  accidents  occurring  during  the  transportation  of 
Army  materiel  by  commercial  carriers  are  not  reportable  through  Army  safety  channels. 
See  Army  Regulation  385-10,  The  Army  Safety  Program,  para.  3-7. h  (Nov.  27,  2013). 
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Figure  10:  Burned  Underside  of  TPS  Carrier’s  Trailer  after  Fire  Incident  and  Undetonated  High  Explosives  inside  Trailer. 


•  In  February  2015,  a  DOD  contractor  conducted  a  preliminary 
investigation  of  an  incident  involving  a  TPS  carrier  transporting 
explosives  that  caught  on  fire  in  September  2014  near  Crawfordville, 
Georgia.  The  tires  and  the  trailer  were  burned  off.  See  figure  1 1 . 
There  were  no  reported  injuries,  but  the  investigation  report  did  not 
identify  whether  there  was  any  damage  to  the  DOD  materials. 
Although  the  report  contained  findings,  conclusions,  and 
recommendations,  the  DOD  contractor  assigned  to  investigate  the 
incident  reported  that  the  root  cause  of  the  incident  was  not 
determined  because  there  was  no  one  required  to  search  for  the  root 
cause  and  that  the  investigation  was  conducted  nearly  four  months 
after  the  incident  had  occurred,  impacting  the  adequacy  and 
sufficiency  of  the  investigation.  Army  Safety  officials  agreed  that  the 
investigation  was  not  completed  because  the  root  cause  was  never 
identified.  In  figure  1 1,  we  provide  a  photograph  of  the  TPS  carrier’s 
trailer  that  caught  on  fire  while  transporting  explosives  near 
Crawfordville,  Georgia,  in  September  2014. 


Source:  Army.  I  GAO-16-82 
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Figure  11:  TPS  Carrier’s  Trailer  Being  Towed  Away  after  Fire  Incident 


Source:  Army.  |  GAO-16-82 


•  In  March  2015,  a  DOD  contractor  conducted  a  preliminary 

investigation  of  a  trailer  fire  that  occurred  in  December  2014  in  Huson, 
Montana,  according  to  DOD  safety  officials.  However,  the  report 
provided  by  DOD  on  the  investigation  did  not  contain  the  root  cause, 
findings  or  conclusions  on  what  caused  the  fire,  or  recommendations 
on  corrective  actions  for  preventing  future  recurrences.  The  report 
primarily  contained  information  on  the  carrier’s  facility  inspection  that 
was  conducted  on  the  same  day. 

Army  Safety  officials  stated  that  they  initiated  preliminary  investigations  of 
the  three  incidents  based  on  an  ad  hoc  approach  versus  a  required 
process  driven  by  department-wide  guidance  that  emphasizes  the 
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importance  of  conducting  a  full  investigation  including  the  identification  of 
the  root  cause.  They  further  stated  that  investigations  should  identify  such 
elements  as  the  root  cause  and  the  factors  contributing  to  the  cause  of 
the  incident.  For  example,  Army  Safety  personnel  stated  that  DOD  did  not 
require  the  DOD  contractor  to  examine  other  factors  during  the 
investigations  such  as  the  training  and  mental  state  of  the  drivers  to 
determine  whether  the  drivers  might  have  contributed  to  the  cause.  An 
Army  briefing  describing  the  investigations  that  the  Army  conducts  in 
other  contexts  indicates  that  human  factors  such  as  training,  individual 
capabilities  and  actions  are  examined  during  an  investigation  to 
determine  the  extent  to  which  they  contributed  to  the  incident. 

We  also  found  that  DOD  is  not  investigating  near  miss  incidents  in  order 
to  identify  the  root  cause  and  identify  corrective  action  that  could  prevent 
future  recurrences  of  these  situations.  DOD  Instruction  6055.07,  Mishap 
Notification,  Investigation,  Reporting,  and  Record  Keeping,  also  briefly 
discusses  near  misses,  which  are  defined  as  an  undesired  event  that, 
under  slightly  different  circumstances,  would  have  resulted  in  personal 
harm,  property  damage,  or  an  undesired  loss  of  resources,  although  it 
does  not  require  investigation  of  near  misses.44  SDDC  officials  stated  that 
there  have  been  a  number  of  TPS  carrier-related  near  misses  that  they 
believed  warranted  investigations  to  determine  the  root  cause  and  safety 
risks.  For  example,  in  September  2014,  the  DOD’s  Transportation 
Inspection  Team  found  a  shipment  of  DOD  small  arms  unattended  at  a 
commercial  truck  stop  during  a  DOD  in  transit  inspection.  A  DOD  truck 
later  came  to  pick  up  the  shipment  and  place  it  in  a  secure  location. 
However,  the  incident  report  did  not  indicate  why  the  DOD  small  arms 
were  unattended  at  a  commercial  truck  stop  because  the  incident  was 
never  investigated  to  determine  the  root  cause  and  safety  risks  to  the 
public.  Nevertheless,  near  misses,  such  as  a  mechanical  failures  leading 
to  fire,  could  lead  to  an  explosion  or  result  in  other  serious  consequences 
and  thus  may  warrant  investigations  to  identify  root  causes  and  prevent 
future  incidents. 

Moreover,  although  the  Defense  Transportation  Tracking  System  collects 
and  tracks  incidents,  the  system  does  not  identify  which  incidents  are 
mishaps  or  near  misses,  which  could  be  used  to  determine  whether 


44  See  Department  of  Defense  Instruction  6055.07,  Mishap  Notification,  Investigation, 
Reporting,  and  Record  Keeping,  Table  7  (June  6,  201 1 );  id.  at  49. 
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further  investigation  was  warranted.  For  example,  the  incident  report  of 
the  Vian,  Oklahoma  incident  in  April  2014  was  identified  as  a  mechanical 
breakdown  rather  than  a  fire  and  it  did  not  indicate  the  severity  of  the 
incident  such  as  whether  it  was  a  mishap  or  near  miss.  SDDC  previously 
modified  the  Defense  Transportation  Tracking  System  to  enable  them  to 
identify  and  address  other  issues.  For  example,  the  Defense 
Transportation  Tracking  System  was  modified  in  November  2014  to 
identify  secure-hold  denials  on  the  incident  reports  based  on  findings  and 
recommendations  from  our  May  2014  report.45  According  to  SDDC 
officials  responsible  for  managing  the  Defense  Transportation  Tracking 
System,  this  enables  DOD  to  track  issues  with  secure-hold  denials  and 
take  corrective  actions. 

DOD  does  not  conduct  full  investigations  of  mishaps  and  investigations  of 
near  misses  involving  security-sensitive  shipments  because  there  is  no 
department-wide  guidance  that  requires  comprehensive  investigations  of 
incidents  involving  commercial  carriers  with  security-sensitive  materials  to 
determine  potential  root  causes  and  identify  corrective  actions  that  could 
mitigate  the  recurrence  of  the  mishap  or  the  potential  for  more  significant 
ones.  In  addition,  DOD  Explosives  Safety  Board  and  Army  Safety  officials 
stated  that  the  guidance  such  as  DOD  Instruction  6055.07,  Mishap 
Notification,  Investigation,  Reporting,  and  Record  Keeping,  on  whether 
and  how  to  initiate  investigations  of  mishaps  is  confusing  and  unclear, 
because  it  excludes  commercial  carriers,  such  as  TPS.  Furthermore, 
according  to  findings  in  a  study  on  the  DOD  Implementation  Plan  for  the 
Distribution  of  Arms,  Ammunition  and  Explosives,46  existing  guidance 
often  leads  to  confusion  and  inconsistent  application  of  procedures 
concerning  aspects  of  safety  and  may  even  result  in  direct  conflict 
between  DOD  organizations.  According  to  DOD  Explosives  Safety  Board 
and  Army  Safety  officials,  many  of  the  safety-related  rules  and  regulations 
that  affect  transportation  of  security-sensitive  materials  are  dispersed  in 
many  publications  and  not  easily  located.  They  further  stated  that  they 
lack  department-wide  guidance  to  ensure  that  DOD  conducts 
comprehensive,  full  investigations  that  consistently  examine  all  the  factors 
to  determine  the  root  cause  of  incidents  involving  TPS  carriers  and 
prevent  future  recurrences. 


45  GAO-14-375. 

46  Department  of  Defense  Implementation  Plan,  Distribution  of  Arms,  Ammunition  and 
Explosives  (March  2005). 
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Without  establishing  a  requirement  for  the  evaluation  of  the  root  causes  of 
collected  incidents  to  identify  systemic  trends  and  patterns  and  full 
investigations  of  incidents,  DOD  is  not  positioned  to  identify  safety  risks 
and  effectively  plan  to  mitigate  and  prevent  recurrences  to  the  extent 
possible.  In  addition,  although  DOD  does  not  require  the  investigation  of 
near  misses  involving  TPS  carriers,  near  misses  such  as  a  mechanical 
failure  leading  to  a  fire  could  ignite  an  explosion  or  result  in  other  serious 
consequences.  Thus,  establishing  department-wide  guidance  to  identify 
and  fully  investigate  incidents,  including  mishaps  and  near  misses 
involving  security-sensitive  materials,  would  help  the  Department  identify 
and  address  their  root  causes  to  prevent  future  recurrences. 


Conclusions 


The  probability  that  a  major  mishap,  such  as  an  accident,  will  occur  in  the 
near  future  that  involves  a  shipment  of  security-sensitive  materials  such 
as  ammunitions  and  explosives  is  unknown.  However,  the  effects  could 
be  devastating  as  the  release  of  hazardous  or  security-sensitive  material 
could  result  in  damage  to  DOD  materials  or  infrastructure  as  well  as 
major  injuries  or  fatalities  that  could  cost  DOD  billions  of  dollars. 

Moreover,  these  shipments  are  at  a  heightened  risk  of  theft  or  of  the 
compromise  of  sensitive  national  security  information.  By  updating  TPS 
program  guidance  to  (1)  address  actions  to  take  when  absent  or  dated 
safety  ratings  and  poor  safety  scores  exist  and  (2)  document  specific 
consequences  to  be  enforced  when  carriers  do  not  meet  program 
requirements,  and  (3)  require  reviews  of  available  violation  data,  DOD 
could  help  ensure  that  it  effectively  evaluates  and  accepts  or  retains 
carriers  entrusted  to  transport  security-sensitive  material  in  the  TPS 
program.  As  described  above,  violation  data  is  a  rich  source  of 
information  about  a  carrier’s  safety  record,  and  can  be  matched  to  TPS 
shipment  data  to  identify  TPS  shipment-specific  violations.  Further, 

DOD’s  actions,  such  as  never  placing  a  carrier  in  non-use  status,  may  not 
be  fully  consistent  with  the  department’s  guidance,  which  indicates  that  a 
carrier  may  be  placed  in  non-use  status  for  failure  to  improve  poor  Safety 
Measurement  System  scores.  This  does  not  necessarily  suggest  that 
DOD  should  be  placing  more  carriers  in  non-use  status,  but  rather  that  it 
should  consider  revising  its  guidance  to  better  match  its  actions. 
Furthermore,  DOD  does  not  currently  require  evaluations  of  incident  data 
from  the  Defense  Transportation  Tracking  System  involving  TPS  carriers 
with  security-sensitive  materials,  and  therefore  is  not  in  a  position  to 
identify  systemic  issues  linked  to  serious  incidents,  as  well  as  with  near 
misses,  that  could  reduce  future  incidents.  Similarly,  DOD  is  not 
identifying  the  root  causes  of  incidents,  which  could  also  help  prevent 
future  ones.  Without  requiring  systematic  evaluations  of  incident  data  and 
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investigations  of  the  root  causes  of  TPS  carrier  incidents,  via 
comprehensive  department-wide  guidance,  DOD  is  not  in  the  best 
position  to  help  protect  TPS  drivers  and  the  public  from  injuries  or 
fatalities  that  may  result  from  incidents  involving  the  transport  of  security- 
sensitive  materials. 


Rprnmmpnrlatinnc;  fnr  Overa^  we  are  making  four  recommendations  related  to  strengthening 

DOD’s  ability  to  effectively  evaluate  carriers  it  contracts  with  to  transport 
Executive  Action  security-sensitive  materials  and  to  mitigate  related  public  safety  risks  that 

may  occur  during  transport  of  those  materials. 

To  help  ensure  that  DOD  effectively  evaluates  the  safety  performance  of 
carriers  entrusted  to  transport  security-sensitive  materials  in  the 
Transportation  Protective  Services  (TPS)  program,  we  recommend  that 
the  Secretary  of  Defense  direct  the  Office  of  the  Under  Secretary  of 
Defense  for  Acquisition,  Technology,  and  Logistics,  in  collaboration  with 
the  U.S.  Transportation  Command  to: 

•  update  TPS  program  guidance  to  clarify  (1)  how  to  address  carriers 
with  absent  or  dated  Safety  Ratings  and  poor  Safety  Measurement 
System  scores,  and  (2)  what  specific  actions  should  be  taken  when 
carriers  do  not  meet  program  requirements 

•  establish  and  document  an  approach  for  conducting  reviews  of 
available  violation  data,  such  as  analyzing  violations  incurred  while 
transporting  TPS  shipments. 

To  improve  DOD’s  ability  to  identify  and  effectively  mitigate  public  safety 
risks  of  TPS  carriers  transporting  security-sensitive  materials,  we 
recommend  that  the  Secretary  of  Defense  direct  the  Office  of  the  Under 
Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics,  in 
collaboration  with  the  DOD  Explosives  Safety  Board,  the  U.S. 
Transportation  Command,  the  Surface  Deployment  and  Distribution 
Command,  and  the  Army  Headquarters  Safety  Office,  to  develop 
department-wide  guidance  requiring  the: 

•  evaluation  of  the  Defense  Transportation  Tracking  System  TPS 
carrier  incident  data  to  identify  trends  and  patterns  that  could  suggest 
systemic  weaknesses  such  as  mechanical  breakdowns  or  unusual 
delays  that  represent  a  heightened  potential  public  safety  risk  and 
take  action  to  address  any  identified  weaknesses. 

•  identification  and  full  investigation  of  TPS  carrier  incidents,  including 
mishaps  and  near  misses  involving  security-sensitive  shipments,  to 
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determine  potential  root  causes  and  identify  corrective  actions  that 
could  mitigate  the  recurrence  of  the  mishap  or  the  potential  for  more 
significant  ones. 

Agency  Comments 
and  Our  Evaluation 

We  provided  a  draft  of  this  report  to  DOD  and  DOT  for  comment.  DOT 
provided  technical  comments,  which  we  incorporated  as  appropriate.  In 
written  comments,  reproduced  in  their  entirety  in  appendix  II,  DOD 
concurred  with  two  of  our  recommendations  and  did  not  concur  with  the 
remaining  two. 

DOD  did  not  concur  with  our  first  recommendation  that  the  Secretary  of 
Defense  direct  the  Office  of  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics,  in  collaboration  with  the  U.S. 
Transportation  Command  to  update  Transportation  Protective  Services 
(TPS)  program  guidance  to  clarify  (1)  how  to  address  carriers  with  absent 
or  dated  Safety  Ratings  and  poor  Safety  Measurement  System  scores, 
and  (2)  what  specific  actions  should  be  taken  when  carriers  do  not  meet 
program  requirements.  In  its  comments,  DOD  stated  that  the  TPS 
program  guidance  is  sufficient  in  describing  carrier  safety  requirements, 
including  the  process  for  taking  action  when  carriers  do  not  meet  program 
requirements.  DOD  stated  that  the  7  TPS  carriers  that  we  identified 
without  Safety  Ratings  are  not  permitted  to  move  ammunition  and 
explosives.  DOD  stated  that  it  uses  a  contractor  to  regularly  evaluate  the 
safety  performance  of  TPS  carriers  and  drivers.  Finally,  DOD  noted  that 
all  TPS  carriers  must  have  Safety  Measurement  System  scores  that  meet 
DOT  requirements  but  that  the  department  has  decreased  its  reliance  on 
these  scores  in  response  to  the  findings  in  our  February  2014  report 
(GAO-14-114)  on  the  scores’  limitations. 

We  continue  to  believe  that  DOD  should  update  its  TPS  guidance  to 
clarify  how  to  address  carriers  with  absent  or  dated  Safety  Ratings  and 
poor  Safety  Measurement  System  scores  and  what  specific  actions 
should  be  taken  when  carriers  do  not  meet  program  requirements.  We 
note  in  our  report  that  DOD  officials  had  told  us  during  the  course  of  our 
review  that  they  had  made  decisions  in  internal  discussions  about 
consequences  for  carriers  (such  as  for  the  7  without  Safety  Ratings)  that 
do  not  meet  program  requirements.  These  DOD  officials  confirmed  that 
these  consequences  are  not  documented  in  guidance.  We  believe  that 
TPS  guidance  should  specify  whether  or  not  carriers  with  absent  Safety 
Ratings  including  the  7  we  identified,  are  permitted  to  move  specific  types 
of  sensitive  shipments,  such  as  ammunition  and  explosives.  While  DOD 
stated  that  the  7  carriers  we  identified  without  safety  ratings  are  not 
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permitted  to  move  specific  types  of  sensitive  shipments,  as  our  report 
notes,  the  guidance  was  not  clear  on  this  point.  Similarly,  regarding 
Safety  Measurement  System  scores,  we  note  in  our  report  that  DOD 
guidance  on  these  scores  does  not  address  which  specific  scores  or 
trends,  such  as  scores  above  required  thresholds  for  a  certain  number  of 
months  or  scores  above  thresholds  in  certain  categories,  would  prompt 
which  defined  consequences.  As  we  note  in  our  report,  DOD  has 
maintained  carriers  in  the  TPS  program  that  do  not  meet  requirements 
regarding  Safety  Measurement  System  scores,  and  during  the  course  of 
our  review  we  found  that  decisions  about  consequences  for  this  were 
based  on  undocumented,  internal  discussions.  By  more  comprehensively 
documenting  in  guidance  which  actions  DOD  should  take  when  TPS 
carriers  do  not  meet  Safety  Rating  and  Safety  Measurement  System 
score  requirements,  including  the  extent  to  which  DOD  might  be  reducing 
its  reliance  Safety  Measurement  System  scores,  DOD  could  more  closely 
align  its  practices  with  its  policy — a  necessary  step  for  helping  to  ensure 
that  program  requirements  are  carried  out  as  intended. 

DOD  did  not  concur  with  our  second  recommendation  that  the  Secretary 
of  Defense  direct  the  Office  of  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics,  in  collaboration  with  the  U.S. 
Transportation  Command  to  establish  and  document  an  approach  for 
conducting  reviews  of  available  violation  data,  such  as  analyzing 
violations  incurred  while  transporting  TPS  shipments.  In  its  comments, 
DOD  stated  that  it  agreed  with  the  value  of  reviewing  violation  data, 
particularly  violations  incurred  while  transporting  TPS  material,  and  that  it 
could  support  a  recommendation  that  would  give  DOD  the  opportunity  to 
assess  the  actions  needed  for  an  executable  violation  data  analysis 
process,  as  well  as  the  time  needed  for  such  an  assessment.  DOD 
identified  three  specific  reasons  for  not  concurring  with  our 
recommendation — specifically,  that  DOD  does  not  own  or  collect  the 
violation  data  (DOT  owns  and  collects  the  data),  that  DOT  data  do  not 
distinguish  between  violations  occurring  on  TPS  shipments  and  violations 
occurring  on  non-TPS  shipments,  and  that  additional  research  is  needed 
on  other  DOT  data  elements. 

We  continue  to  believe  that  DOD  should  establish  and  document  an 
approach  for  conducting  reviews  of  available  violation  data,  such  as 
analyzing  violations  incurred  while  transporting  TPS  shipments.  As  we 
note  in  our  report,  most  of  the  violation  data  that  we  refer  to  are  publicly 
available  on  DOT’S  Compliance,  Safety,  Accountability  program  website 
and  could  be  used  to  analyze  meaningful  trends  within  and  across 
carriers,  particularly  for  carriers  that  do  not  have  Safety  Measurement 
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System  scores.  It  is  important  that  DOD  consider  the  safety  performance 
of  an  entire  carrier,  not  just  of  those  vehicles  transporting  freight  under 
the  TPS  program,  because  the  safety  record  of  a  carrier  provides 
important  information  that  reflects  that  carrier’s  safety  policies  and  culture. 
Like  DOD,  we  recognize  that,  as  presented  on  the  DOT  website,  the 
violation  data  do  not  distinguish  between  TPS  and  non-TPS  shipments. 
However,  as  we  stated  in  our  report,  a  senior  DOD  official  told  us  during 
the  course  of  our  review  that  it  would  be  possible  to  use  that  violation 
data  to  identify  specific  TPS  shipments  by  matching  vehicle  identification 
numbers  (VIN)  with  dates  for  TPS  shipments  to  determine  whether 
vehicles  transporting  TPS  shipments  had  received  violations  and,  if  so, 
what  those  violations  were.  Finally,  we  agree  with  DOD  about  the  need  to 
dedicate  time  to  assess  the  actions  needed  to  develop  a  process  for 
analyzing  violation  data,  and  believe  our  recommendation  allows  DOD 
the  flexibility  to  determine  the  feasibility  of  such  a  process. 

DOD  concurred  with  our  third  and  fourth  recommendations  that,  to 
improve  DOD’s  ability  to  identify  and  effectively  mitigate  public  safety 
risks  of  TPS  carriers  transporting  security-sensitive  materials,  the 
Secretary  of  Defense  direct  the  Office  of  the  Under  Secretary  of  Defense 
for  Acquisition,  Technology,  and  Logistics,  in  collaboration  with  the  DOD 
Explosives  Safety  Board,  the  U.S.  Transportation  Command,  the  Surface 
Deployment  and  Distribution  Command,  and  the  Army  Headquarters 
Safety  Office,  to  develop  department-wide  guidance  requiring  (1)  the 
evaluation  of  the  Defense  Transportation  Tracking  System  TPS  carrier 
incident  data  to  identify  trends  and  patterns  that  could  suggest  systemic 
weaknesses  such  as  mechanical  breakdowns  or  unusual  delays  that 
represent  a  heightened  potential  public  safety  risk  and  take  action  to 
address  any  identified  weaknesses;  and  (2)  the  identification  and  full 
investigation  of  TPS  carrier  incidents,  including  mishaps  and  near  misses 
involving  security-sensitive  shipments,  to  determine  potential  root  causes 
and  identify  corrective  actions  that  could  mitigate  the  recurrence  of  the 
mishap  or  the  potential  for  more  significant  ones.  DOD  did  not  identify  any 
specific  steps  that  it  planned  to  take  to  address  these  two 
recommendations. 


We  are  sending  copies  of  this  report  to  the  appropriate  congressional 
committees  and  to  the  Secretary  of  Defense  and  the  Secretary  of 
Transportation.  The  report  also  is  available  at  no  charge  on  GAO’s 
website  at  http://www.gao.gov. 
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If  you  or  your  staff  have  any  questions  about  this  report,  please  contact 
Cary  Russell  at  (202)  512-5431  or  russellc@gao.gov;  or  Susan  Fleming 
at  (202)  512-2834  or  flemings@gao.gov.  Contact  points  for  our  Offices  of 
Congressional  Relations  and  Public  Affairs  may  be  found  on  the  last  page 
of  this  report.  Key  contributors  to  this  report  are  listed  in  appendix  III. 


Director 

Defense  Capabilities  and  Management 


Susan  A.  Fleming 
Director 

Physical  Infrastructure  Issues 
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Appendix  I:  Scope  and  Methodology 


To  address  the  extent  to  which  the  Department  of  Defense’s  (DOD)  use 
of  the  Department  of  Transportation’s  (DOT)  Compliance,  Safety, 
Accountability  (CSA)  program  data  and  other  safety  performance 
information  results  in  sufficient  and  reliable  information  for  DOD  to 
evaluate  the  safety  performance  of  individual  Transportation  Protective 
Services  (TPS)  carriers,  we  reviewed  DOD  guidance  regarding  DOD’s 
procedures  for  evaluating  the  safety  of  TPS  carriers  (i.e.,  for  them  to  enter 
and  remain  in  the  TPS  program)  and  findings  from  our  February  2014 
report.1  We  assessed  DOD  guidance  against  Standards  for  Internal 
Control  in  the  Federal  Government.2  We  also  reviewed  DOT  safety 
performance  information,  including  Safety  Measurement  System  scores 
and  Safety  Ratings.  We  reviewed  DOT  safety  performance  information 
available  for  the  55  commercial  motor  carriers  that  participated  in  the  TPS 
program  from  fiscal  years  201 1  through  2014,  which  were  the  most  recent 
available  data.  To  determine  the  extent  to  which  the  limitations  we  had 
reported  on  in  February  2014  regarding  Safety  Measurement  System 
scores  applied  specifically  to  individual  TPS  carriers,  we  obtained  and 
analyzed  the  most  recently  available  2-year  “snapshot”  of  CSA  safety 
performance  data  (November  2012  through  October  2014).  The  2-year 
snapshots  are  the  basis  for  DOT’S  score  calculations.  At  some  point 
during  this  period,  2  of  the  55  TPS  carriers  ceased  activity;  therefore,  we 
excluded  those  carriers  from  any  analyses  that  required  observations 
over  the  entire  2-year  period. 

In  addition,  we  compared  TPS  carriers’  Safety  Measurement  System 
scores  in  this  2-year  snapshot  to  those  DOD  requires  carriers  to  maintain 
to  remain  in  the  TPS  program.  We  were  unable  to  compare  TPS  carriers’ 
Safety  Measurement  System  scores  at  entry  into  the  TPS  program  with 
those  DOD  requires  of  carriers  to  enter  the  TPS  program  because  the 
CSA  Program — including  Safety  Measurement  System  scores — was  not 
implemented  until  2010,  and  most  TPS  carriers  entered  the  program 
before  then.  To  evaluate  DOD’s  use  of  Safety  Ratings,  which  are 
comprehensive  assessments  of  a  carrier’s  compliance  with  safety  fitness 
standards  established  in  DOT  regulations  that  FMCSA  conducts 
periodically,  we  first  examined  the  most  recent  Safety  Rating  data 
available  for  all  55  TPS  carriers  within  our  November  2012  through 


1 GAO-14-1 14. 

2  GAO,  Standards  for  Internal  Control  in  the  Federal  Government,  GAO/AIMD-OO-21.3.1 
(November  1999). 
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October  2014  data  set,  which  was  October  2014.  We  compared  DOD’s 
use  of  DOT  safety  performance  information  with  GAO’s  standards  for 
data  reliability3  and  with  Standards  for  Internal  Controls  in  the  Federal 
Government.  In  addition,  we  met  with  officials  from  DOD’s  U.S. 
Transportation  Command  to  identify  key  components  of  the  TPS  program 
and  with  officials  from  the  Department  of  Energy’s  Environmental 
Management  Program  to  compare  their  program  that  uses  commercial 
motor  carriers  to  transport  some  hazardous  materials  to  the  TPS 
program,  and  the  American  Trucking  Associations,  to  identify  information 
on  safety  data  or  standards  recommended  or  used  by  other  agencies  and 
organizations  knowledgeable  about  the  transport  of  security-sensitive 
materials.  We  determined  that  the  DOT  safety  performance  information 
was  sufficiently  reliable  for  the  purpose  of  our  data  analysis  by  reviewing 
relevant  DOT  documentation  on  Safety  Measurement  System 
methodology,  performing  standard  electronic  data  reliable  testing  such  as 
looking  for  outliers  or  missing  data  in  the  2012-14  data  for  TPS  carriers, 
and  interviewing  FMCSA  officials  about  any  differences  in  Safety 
Measurement  System  methodology  or  additional  data  for  hazardous 
materials  carriers.  Specifically,  the  purpose  of  our  analysis  was  to 
determine  the  extent  to  which  limitations  in  Safety  Measurement  System 
methodology  we  identified  in  our  February  2014  (GAO-14-1 14)  report 
applied  to  all  55  TPS  carriers. 

To  examine  the  extent  to  which  DOD  has  evaluated  TPS  carrier  incident 
data  and  investigated  TPS  carrier  incidents  to  identify  safety-related 
trends,  we  compared  safety  processes  and  procedures  from  DOD 
Instruction  6055.16,  on  DOD’s  Explosives  Safety  Management  Program, 
with  our  analysis  of  1 ,039  incident  reports  from  the  Defense 
Transportation  Tracking  System  from  fiscal  year  201 1  through  fiscal  year 
2014,  to  determine  what  information  DOD  collects,  reports,  and  tracks 
when  there  is  an  incident  (i.e.,  an  accident,  mechanical  breakdown,  or 
suspicious  activity)  reported  through  the  Defense  Transportation  Tracking 
System  involving  TPS  carriers.  We  also  compared  DOD’s  processes  and 
procedures  for  analyzing  incident  data  and  investigating  incidents  with  the 
DOD  Instruction  6055.16,  which  emphasizes  the  importance  of 
conducting  appropriate  analyses  of  incidents  involving  DOD  ammunitions 
and  explosives  to  preventing  future  incidents.  For  the  purpose  of  our 


3  GAO,  Assessing  the  Reliability  of  Computer-Processed  Data,  GAO-09-365G  (February 
2009). 
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review,  we  defined  an  “incident”  as  an  accident,  mechanical  breakdown, 
suspicious  activities  or  other  event  reported  through  the  Defense 
Transportation  Tracking  System  for  emergency  response  purposes.  We 
defined  “mishap”  and  “near  miss  incidents”  based  on  the  definitions  in 
DOD  Instruction  6055. 07. 4  We  interviewed  officials  from  the  Office  of  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics; 
the  DOD  Explosives  Safety  Board;  the  U.S.  Transportation  Command; 
and  the  Army  Headquarters  Safety  Office.  We  reviewed  studies  and 
reports  from  organizations  such  as  the  National  Transportation  Safety 
Board,  National  Trucking  Associations,  and  the  American  Trucking 
Association  on  recommended  technologies  and  safety  standards  for  TPS 
carriers.  We  assessed  the  reliability  of  the  Defense  Transportation 
Tracking  System  data  and  determined  the  data  we  examined  were 
sufficiently  reliable  for  identifying  challenges  and  the  extent  to  which  they 
impact  the  transport  of  security-sensitive  materials.  We  sent  a 
questionnaire  to  the  Defense  Transportation  Tracking  System  officials 
who  maintain  and  provide  oversight  over  the  system  about  the  reliability 
of  the  data  as  well  as  other  internal  and  quality  controls  in  place.  We 
followed  up  with  interviews  as  needed  to  clarify  any  responses.  We  also 
requested  and  reviewed  documentation  about  how  the  system  is 
structured,  the  data  dictionary,  and  written  procedures  in  place  to  ensure 
that  the  appropriate  information  in  the  Defense  Transportation  Tracking 
System  is  collected.  Based  on  our  assessment  of  the  reliability  of  the 
Defense  Transportation  Tracking  System  data,  we  determined  that  the 
data  we  examined  were  sufficiently  reliable  for  identifying  challenges  and 
the  extent  to  which  they  impact  the  transport  of  security-sensitive 
materials. 

We  visited  or  contacted  officials  from  the  following  organizations  during 
our  review: 

•  Headquarters,  Defense  Logistics  Agency,  Fort  Belvoir,  Virginia; 


4  Department  of  Defense  (DOD)  Instruction  6055.07  defines  a  near  miss  as  an  undesired 
event  that,  under  slightly  different  circumstances,  would  have  resulted  in  personal  harm, 
property  damage,  or  an  undesired  loss  of  resources.  It  defines  a  mishap  as  an  unplanned 
event  or  series  of  events  that  results  in  damage  to  DOD  property;  occupational  illness  to 
DoD  personnel;  injury  to  on-  or  off-duty  DOD  military  personnel;  injury  to  on-duty  DOD 
civilian  personnel;  or  damage  to  public  or  private  property,  or  injury  or  illness  to  non-DOD 
personnel,  caused  by  DOD  activities.  See  Department  of  Defense  Instruction  6055.07, 
Mishap  Notification ,  Investigation ,  Reporting ,  and  Record  Keeping  at  47,  49  (June  6, 
2011). 
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•  Headquarters,  Army  Safety  Office,  Fort  Belvoir,  Virginia; 

•  Office  of  the  Under  Secretary  of  Defense  (Acquisition,  Technology 
and  Logistics),  Office  of  the  Deputy  Assistant  Secretary  of  Defense 
(Transportation  Policy),  Mark  Center,  Alexandria,  Virginia; 

•  U.S.  Transportation  Command,  Scott  Air  Force  Base,  Illinois; 

•  Surface  Deployment  and  Distribution  Command,  Scott  Air  Force 
Base,  Illinois; 

•  Defense  Transportation  Tracking  System  Office,  Scott  Air  Force  Base, 
Illinois; 

•  Department  of  Energy,  Office  of  Environmental  Management, 
Washington,  D.C.; 

•  U.S.  Department  of  Transportation,  Washington,  D.C.; 

•  Federal  Motor  Carrier  Safety  Administration,  Washington,  D.C.  ; 

•  National  Transportation  Safety  Board,  Washington,  D.C.; 

•  Commercial  Vehicle  Safety  Alliance,  Greenbelt,  Maryland; 

•  American  Trucking  Associations,  Arlington,  Virginia;  and 

•  Owner  Operator  Independent  Drivers  Association,  Washington,  D.C. 

We  conducted  this  performance  audit  from  July  2014  to  December  2015 
in  accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 
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ASSISTANT  SECRETARY  OF  DEFENSE 

3000  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-3000 

OCT  2  8  2015 


LOGISTICS  AND 
MATERIEL  READINESS 


Mr.  Cary  Russell 

Director,  Defense  Capabilities  and  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  N.W. 

Washington,  DC  20548 

Dear  Mr.  Russell: 


This  is  the  Department  of  Defense  (DoD)  response  to  the  Government  Accountability 
Office  (GAO)  Draft  Report,  GAO-16-82,  “DEFENSE  TRANSPORTATION:  DoD  Needs  to 


Improve  the  Evaluation  of  Safety  and  Performance  Information  for  Carriers  Transporting 
Security-Sensitive  Materials,”  dated  September  11, 2015  (GAO  Code  351944).”  Detailed 
comments  on  the  report  recommendations  are  enclosed.  Thank  you  for  the  opportunity  to  provide 
comments  on  this  draft  report.  , 


Enclosure: 
As  stated 
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GAO  DRAFT  REPORT  DATED  SEPTEMBER  11, 2015 
GAO-16-82  (GAO  CODE  351944) 

“DEFENSE  TRANSPORTATION:  DOD  NEEDS  TO  IMPROVE  THE  EVALUATION 
OF  SAFETY  AND  PERFORMANCE  INFORMATION  FOR  CARRIERS 
TRANSPORTING  SECURITY-SENSITIVE  MATERIALS” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATION 

RECOMMENDATION  1:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  the 
Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics,  in 
collaboration  with  the  U.S.  Transportation  Command  to:  update  Transportation  Protective 
Services  (TPS)  program  guidance  to  clarify  (1)  how  to  address  carriers  with  absent  or  dated 
safety  ratings  and  poor  safety  scores,  and  (2)  what  specific  actions  should  be  taken  when  carriers 
do  not  meet  program  requirements. 

DoD  RESPONSE:  Non-concur  with  comment. 

The  TPS  program  guidance  is  sufficient  in  describing  carrier  safety  requirements,  including  the 
Department’s  process  for  taking  action  when  carriers  do  not  meet  program  requirements.  All 
TPS  carriers  approved  to  move  ammunition  and  explosives  have  satisfactory  DoT  safety  ratings. 
The  7  TPS  carriers  that  GAO  identified  without  DoT  safety  ratings  are  not  permitted  to  move 
ammunition  and  explosives,  which  could  pose  a  safety  risk  to  the  public. 

The  DoD  regularly  evaluates  carrier  safety  performance  for  continued  participation  in  the  TPS 
program  through  the  Military  Surface  Deployment  and  Distribution  Command’s  (SDDC) 
Transportation  Safety  and  Security  program  where  a  contractor  performs  reviews,  at  both  carrier 
facilities  and  while  shipments  are  in-transit,  of  carrier  and  driver  safety  performance. 

All  TPS  carriers  must  also  have  safety  scores  that  meet  DoT  requirements.  DoD  has  placed  less 
reliance  on  these  scores  for  evaluating  overall  carrier  safety  performance  as  a  result  of  the  GAO 
Report  14-114,  “Federal  Motor  Carrier  Safety:  Modifying  Compliance,  Safety,  Accountability 
Program  Would  Improve  the  Ability  to  Identify  High  Risk  Carriers,”  dated  February  2014, 
which  was  critical  of  the  DoT  scoring  system  in  a  number  of  areas,  to  include  a  finding  that  the 
majority  of  regulations  used  to  calculate  the  safety  scores  are  not  violated  often  enough  to 
strongly  associate  them  with  crash  risk  for  individual  carriers. 

RECOMMENDATION  2:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  the 
Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics,  in 
collaboration  with  the  U.S.  Transportation  Command  to  establish  and  document  an  approach  for 
conducting  reviews  of  available  violation  data,  such  as  analyzing  violations  incurred  while 
transporting  TPS  shipments. 

DoD  RESPONSE:  Non-concur  with  comment. 
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While  the  Department  agrees  with  the  value  of  conducting  reviews  of  violation  data,  particularly 
violations  incurred  while  transporting  DoD  TPS  material,  the  Department  cannot  concur  with  the 
recommendation  as  written,  for  a  number  of  reasons,  including  the  following: 

•  The  Department  does  not  own  or  collect  the  violation  data;  this  is  done  by  the 
Department  of  Transportation  (DoT). 

•  The  DoT  violation  data  do  not  distinguish  between  violations  occurring  on  DoD  and  non- 
DoD  shipments  or  between  violations  occurring  on  TPS  shipments  and  general  freight 
shipments. 

•  Additional  research  is  needed  on  other  data  elements  in  the  DoT  data. 

The  Department  could  support  a  GAO  recommendation  that  would  give  DoD  the  opportunity  to 
assess  the  actions  needed  for  an  executable  violation  data  analysis  process,  as  well  as  the  time 
needed  for  such  an  assessment. 

RECOMMENDATION  3:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  the 
Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics,  in 
collaboration  with  the  DOD  Explosives  Safety  Board,  the  U.S.  Transportation  Command,  the 
Surface  Deployment  and  Distribution  Command,  and  the  Army  Headquarters  Safety  Office,  to 
develop  department-wide  guidance  requiring:  the  evaluation  of  the  Defense  Transportation 
Tracking  System  TPS  carrier  incident  data  to  identify  trends  and  patterns  that  could  suggest 
systemic  weaknesses  such  as  mechanical  breakdowns  or  unusual  delays  that  represent  a 
heightened  potential  public  safety  risk  and  take  action  to  address  any  identified  weaknesses. 

DoD  RESPONSE:  Concur 

RECOMMENDATION  4:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  the 
Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics,  in 
collaboration  with  the  DOD  Explosives  Safety  Board,  the  U.S.  Transportation  Command,  the 
Surface  Deployment  and  Distribution  Command,  and  the  Army  Headquarters  Safety  Office,  to 
develop  department-wide  guidance  requiring  the  identification  and  full  investigation  of  TPS 
carrier  incidents,  including  mishaps  and  near  misses  involving  security-sensitive  shipments,  to 
determine  potential  root  causes  and  identify  corrective  actions  that  could  mitigate  the  recurrence 
of  the  mishap  or  the  potential  for  more  significant  ones. 

DoD  RESPONSE:  Concur 
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GAO  Contacts 

Cary  Russell,  (202)  512-5431  or  russellc@gao.gov,  or  Susan  Fleming, 
(202)  512-2834  or  flemings@gao.gov 

Staff 

Acknowledgments 

In  addition  to  the  contacts  named  above,  James  Reynolds,  Assistant 
Director:  Brandon  Haller,  Assistant  Director;  Ani  Antanesyan,  Tracy 
Burney,  Sarah  Farkas,  Alfonso  Garcia,  Mae  Jones,  Grant  Mallie,  Terry 
Richardson,  and  Michael  Shaughnessy  made  key  contributions  to  this 
report. 

(351944) 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and 
policies;  and  provides  analyses,  recommendations,  and  other  assistance 
to  help  Congress  make  informed  oversight,  policy,  and  funding  decisions. 
GAO’s  commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no 
cost  is  through  GAO’s  website  (http://www.gao.gov).  Each  weekday 
afternoon,  GAO  posts  on  its  website  newly  released  reports,  testimony, 
and  correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted 
products,  go  to  http://www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO’s  website, 
http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 
MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

Connect  with  GAO 

Connect  with  GAO  on  Facebook,  Flickr,  Twitter,  and  YouTube. 

Subscribe  to  our  RSS  Feeds  or  E-mail  Updates. 

Listen  to  our  Podcasts  and  read  The  Watchblog. 

Visit  GAO  on  the  web  at  www.gao.gov. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Website:  http://www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Katherine  Siggerud,  Managing  Director,  siggerudk@gao.gov,  (202)  512- 
4400,  U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room 

71 25,  Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngd@gao.gov,  (202)512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 
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